I didn't got the meaning or the difference between sparse and dense corpra here in this sentence "the reason is that Skip-gram works better over sparse corpora like Twitter and NIPS, while CBOW works better over dense corpora " enter image description here


1 Answer 1


TL;DR: I would ignore them.

I research this area and have no idea what that means. "Sparse" and "dense" often refer to types of embeddings, the former refers to count-based embeddings (formed of word co-occurrence counts), which are typically sparse as many word pairs do not appear together, the latter refers to neural embeddings, e.g. word2vec, GloVe, etc.

If there is some formal definition of sparse or dense text (based on some kind of information theoretic measure?), then it should be defined or referenced. Note that the wording above is different to the same work in this (presumably) peer-reviewed conference paper. (I don't know which came first).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.