5
$\begingroup$

I am building a deep learning model for NLP. I am pretty comfortable with adding word embedding from word2vec or Glove vectors as extra word features but I wanted to add other word features like POS tag of a word, NER tag of word along with embedding as features. How can I do this. Should I give these word features by concatenating their vector with the word vectors. Or is there some other method. Please suggest.

$\endgroup$
  • 1
    $\begingroup$ Yes, concatenate them. $\endgroup$ – Emre Apr 28 '17 at 21:52
1
$\begingroup$

One option is to concatenate them, the second is to treat them as separate inputs. For example Keras offers such neural model: https://keras.io/getting-started/functional-api-guide/#multi-input-and-multi-output-models

$\endgroup$
0
$\begingroup$

I would concatenate them into a single input vector. Essentially, your model treats each latent variable from the word embedding as a single feature (think about a regular ML model). Adding a couple to the end of this wouldn't hurt your performance too much.

Another option is to follow what @djstrong said, about multi-inputs. But I would start with just concatenating the extra variables at the end of your input vector.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.