I am training a big corpus using word2vec and averaging the word vectors to get sentence vectors. What is the best way to address negative words so that negative and positive sentences are away from each other? For e.g.: "After the fix code worked" and "After the fix code did not work" should ideally give sentence vectors which are far from each other. I heard one approach is to look for negative words like "not" and negate the next word vector. Can someone please clarify if that's a good approach or can suggest a better approach?


3 Answers 3


When you look at the vectors that word2vec generates - negative words may have unique features but can be treated just like positive words. That is to say, as far as the NN is concerned - these are just similar words. You may have to construct "concept vectors" on top of the word vectors to do what you would like to do.

Your parts of speech tagging should automatically mark negating words as ADV. You can then train on these adverbs in conjunction to your verbs as a positive or negative output. Here's an example using spacy:-

import spacy

nlp = spacy.load('en')        # this can take a while
sample_text = u'Do not go.'
parsed_text = nlp(sample_text)
token_text = [token.orth_ for token in parsed_text]
token_pos = [token.pos_ for token in parsed_text]

At this point token_text will be a list of your words and token_pos will be the POS tagging:-

not - ADV
go - VERB

As you can see, "not" is tagged as ADV here. You can now feed this tagged output (or a better parse tree) into a second network to train for a negative or positive output.

Hope this helps.


There is a possibility of refining word2vec vectors, which as research shows capture both, semantic relatedness and semantic similarity, in such a way, that they would capture the relations between words such as antonymy or negation. You can take a look at Counter-Fitting method (or methods in it's related work). Their implementation should be available online. This may improve results of your sentiment analysis method.


You can see this paper Querying Word Embeddings for Similarity and Relatedness.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.