I am aware that LSI, RRI and word embeddings are distributional semantics models. However, I am not certain if the below mentioned are also distributional semantic models.

  • Non-Negative Tensor Factorisation
  • Singular Value Decomposition (SVD)
  • Vector Space Model (VSM)

Please let me know if the above mentioned algorithms are also distributional semantics models. Moreover, please also let me know the other distributional semantics based algorithms.


The main idea here is: "birds of a feather flock together" That is, words that appear near each other inform the "function" of a word.

More importantly, I think of the techniques you mentioned as "methods," rather than "models." The reason why is it seems possible to violate the definition of what a distributional semantic model is without appropriate data preprocessing.

SVD for example, is typically a dimensionality reduction technique, or the pre-cursor to a clustering technique depending on feature engineering and usage of the methodology. In this case, if you were counting the co-occurrence of words across documents -- rows = documents, cols = words, cells = # of times that word appeared in that particular document -- and then you ran SVD on that, you might have the pre-cursor to a "model" that may eventually be something you can call distributional semantic.

Another example could be Word2Vec, where one typically uses a neural network to train a shallow layer, and extract the weights. Word2Vec can be trained via Skip-gram, or continuous bag of words. Since the "meaning" of a word is derived from the co-occurrence and/or proximity to neighboring words, it may be considered a distributional semantic model. FastText is probably even more so, since it explicitly uses the distribution of words in documents to perform a similar vector operation.

Latent Dirichlet Allocation may be another example. Perhaps even Naive Bayes, if used in the appropriate manner.

So ultimately, the answer is, it depends on data pre-processing/feature-engineering and usage, rather than the technique.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.