Say I have $N$ categorical features $f_i$ $i\in(1,N)$ each of which of different alphabet size $n_i$. How can I efficiently optimize the hashing trick on that feature vector? Should I enumerate hash bit length for each feature independently (greedy)? Should i assume no hashing is necessary for features with small alphabet ($n_i$ is small)? What is the best practice strategy?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
How can I efficiently optimize the hashing trick on that feature vector?
Use different size of hash, and hash function and see which work best. One cannot tell in advance.
Should I enumerate hash bit length for each feature independently (greedy)?
No, this is NEVER a good practice. The hash should be common to all features.
Should I assume no hashing is necessary for features with small alphabet (ni is small)?
You can always choose not to hash specific features. It implies you think these features are more important, and should avoid collision.
-
2$\begingroup$ doesn't your third answer ("should I assume no hashing ...") contradicts the second one ("should I enumerate bit length for each feature...")?! if some features are more "important" and "diverse" they might need a higher hashing bit-length representation. $\endgroup$ Jul 28, 2016 at 8:34