0
$\begingroup$

My question is what kind of machine learning models could be used in the case we need to predict something from a large pool of possibilities. For example in a kaggle competition( Instacart challenge ) , they ask to predict what products will a user reorder , based on prior orders.

In this case the spectrum of possible outputs is huge ( there are +40000 possible products )

I have studied some models like CNN and usually the output for this models, is a logits tensor holding the probabilities, the size of the tensor usually corresponds to the possibilities in the classification problem. That are generally small ( maximum something like a hundred in cifar-100

But in this case, seems to me that this is not a good approach, because of the number of possible items.

So this CNN models are appropriated in this kind of tasks? what other approaches could be used?

Thanks.

$\endgroup$
2
$\begingroup$

There is something called "sampled softmax" (e.g. tensorflow's implementation), which simply partitions the output space and at each training step only takes into account one of the partitions (see section 3 from this article to learn the math). Sampled softmax is only meant to speed up training for very large output spaces; at inference time you use normal softmax.

Also, if your output is of hierarchical nature, you can use hierarchical softmax. See this blog post for a review of different methods to handle very large categorical space.

| improve this answer | |
$\endgroup$
0
$\begingroup$

As in your example, +40000 possible products or outputs are kind of very large for all types of algorithms. You probably need to be more innovative than using only one approach for that. e.g. you need to define a distance function for such outputs according to your problem. As it is clear, regular distance functions such as Euclidean, doesn't seem to make sense in the context of large vectors and your specific problem. But as a suggestion you may want to read about Structured support vector machine and Random forest. Also you may find other probabilistic approaches interesting too, e.g. HMM.

All in all, we may not search for an absolute answer specially in real-world problems and datasets as in your defined problem. Understanding the data and the related features will help you in order to find the best solution.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.