- Imagine the process of count vectorizer: you first create a vocabulary which maps each word (or n-gram) to an integer index (index in the document term matrix). Then, for each document, you count number of times a word appears and set that value at appropriate index to build vector representation for the document.
- This can potentially create a very large number of features since each n-gram/token is one feature.
- Even if you want to limit the total number of features by using some trick like top-N words by occurrence, you still need to calculate and hold in memory the map of all word-counts. This can be potentially prohibitive in some applications.
- Similar problem happens for TfIDf, where you additionally store the mapping of word to document occurance for calculating the IDf part.
- Either way, you are doing multiple passes over the data and/or potentially large amount of memory consumption.
- The problem is also with bounds or predictability: you do not know the potential memory usage upfront in first phase.
- Hashing vectorizer can build document representation for all documents in one single pass over the data and still keep memory bounded (not necessarily small, the size depends on size of hash-table).
- In a single pass, you calculate hash of a token. Based on the hash value, you increment the count of particular index in the hash-table (the array underlying the hash table implementation). You get representation of current document without looking at every other document in the corpus.
- This gives rise to problem with representation accuracy. Two different tokens may have hash collision.
So you are in effect trading [representation accuracy and explanatory power] Vs. [space (bounded predictable memory usage) and time (no multiple passes on the data)].
Answers to your specific questions
- There is a (sort) correlation between input words and features: through the hash function. But this correlation is potentially defective (hash-collisions) and there's no inverse transformation (you can't say what word is represented by feature number 207).
- There's no fit and transform. For a fixed hash-function, no dataset specific learning is happening (ala word2vec).
- There's no semantic interpretation of the distance. Two words semantically similar words may not be close to each other in the representation. As long as two almost (syntactically, based on tokens) similar documents are close enough, it will work on text classification.
Why Would It Work?
Given these information, you are right in being skeptical: why on earth this should work? The answer is empirical: a randomized representation like hashing works reasonably well in practice (the benefits from exact count based representation are not that great). There might be some theoretical explanation too but I don't know it enough. If curious, you can probably read up this paper.