I have a 2M instances dataset with millions of very very sparse dummy variables created using the
hashing trick =
hash(orig_feature_name + orig_feature_value)=1. Note that the data is sparse both on rows (every instance has only a limited <100 features=1) and on columns (most features are relevant only to very few instances < 1%)
But I'm not sure why shouldn't I prefer a
batch gradient descent algorithm? FTRL for all its merits is still an
online-learning algorithm that sees one instance at a time.
My intuition is that if possible to use all the data for each iteration, we should do it, but I'm not sure...