Literature on selecting specific dimensions in a word embedding vector

Question

I am aware that the different dimensions in the word embedding represents different information and algebraic operations can be performed between two embeddings for example.

Can anyone point me to literature on selecting specific dimensions from a word embedding vector. I don't mean dimensionality reduction, but papers following the theory that all those dimensions are not important for all the tasks and that specific ones will be more important for a specific task. For example: Sentiment Analysis would benefit from dimension 5th to 55th and 250th to 300th instead of using the whole 300 dimensions.

Let me know if this theory isn't true.

Answer 1

There is not much theoretical work on the topic. The number of dimensions for an embedding is an empirical question depending on the corpus and the task.

The "GloVe: Global Vectors for Word Representation" paper by Pennington showed that there are large gains in accuracy by adding dimensions up to couple of hundred then diminishing returns.

The most common number of dimensions is either 128 or 256. 128 and 256 are both powers of 2 which can speed up training time.