I have been working on a trained data for Word2vec algorithm. Since we need words to stay as original we don't make them lowercase at the preprocessing phase. Thus there are words with different variations (e.g "Earth" and "earth").
The only way I can think of is to take average of vectors for "Earth" and "earth" to create a single vector to represent the word. (Since the feature vector's dimensions are similar)
Is this an "okay" method? If it is not, what might be a good way to handle this issue?
Note: Lowering all words in preprocessing is not an option for now.
Edit: The info about whether or not feature dimensions are truly linear would also be helpful.
Edit 2: Combining both answers from
yazhi gave the best results. How are these combined? Weighted average improved the results but putting word frequencies through a scaled sigmoid function gave the best results, because using word frequencies in a linear manner gives them more importance than they bear.