How to interpret Hashingvectorizer representation?

Question

I cannot really understand the logic behind Hashingvectorizer for text feature extraction. I can follow the logic of Bag of Word or TFiDF where the features are values for all/certain words/N-grams per document and as such one can compute (dis)similarity between the representation vector.

1. But how could one imagine the arbitrary number of features in case of Hashing?
2. Are these features purely synthetic features with no correlation of the original words?
3. Is it able to learn from some training set (by "fit") and apply this learned features to new sentences (by "transform") like other vectorizers?
4. Does Hashing represent any semantic distance like word2vec for words? How?

Answer 1

Basic Background

1. Imagine the process of count vectorizer: you first create a vocabulary which maps each word (or n-gram) to an integer index (index in the document term matrix). Then, for each document, you count number of times a word appears and set that value at appropriate index to build vector representation for the document.
2. This can potentially create a very large number of features since each n-gram/token is one feature.
3. Even if you want to limit the total number of features by using some trick like top-N words by occurrence, you still need to calculate and hold in memory the map of all word-counts. This can be potentially prohibitive in some applications.
4. Similar problem happens for TfIDf, where you additionally store the mapping of word to document occurance for calculating the IDf part.
5. Either way, you are doing multiple passes over the data and/or potentially large amount of memory consumption.
6. The problem is also with bounds or predictability: you do not know the potential memory usage upfront in first phase.
7. Hashing vectorizer can build document representation for all documents in one single pass over the data and still keep memory bounded (not necessarily small, the size depends on size of hash-table).
8. In a single pass, you calculate hash of a token. Based on the hash value, you increment the count of particular index in the hash-table (the array underlying the hash table implementation). You get representation of current document without looking at every other document in the corpus.
9. This gives rise to problem with representation accuracy. Two different tokens may have hash collision.

So you are in effect trading [representation accuracy and explanatory power] Vs. [space (bounded predictable memory usage) and time (no multiple passes on the data)].

Answers to your specific questions

1. There is a (sort) correlation between input words and features: through the hash function. But this correlation is potentially defective (hash-collisions) and there's no inverse transformation (you can't say what word is represented by feature number 207).
2. There's no fit and transform. For a fixed hash-function, no dataset specific learning is happening (ala word2vec).
3. There's no semantic interpretation of the distance. Two words semantically similar words may not be close to each other in the representation. As long as two almost (syntactically, based on tokens) similar documents are close enough, it will work on text classification.

Why Would It Work?

Given these information, you are right in being skeptical: why on earth this should work? The answer is empirical: a randomized representation like hashing works reasonably well in practice (the benefits from exact count based representation are not that great). There might be some theoretical explanation too but I don't know it enough. If curious, you can probably read up this paper.