You are correct that recommender systems that map similarity to distance is useful.
Vector representations are useful because most machine learning learning tools are based on linear algebra. Vector representations encode raw data in form that amenable to machine learning.
"Any" vector representation is more useful than no vector representation. ...
You can extract a random sample from the whole data set.
Start with a very small one (~1000 rows) to build a first recommendation system. Then you can increase the quantity depending on the results and the computation capabilities. If you have new data, just repeat the same process.
There are many well-documented techniques to help you out with this. Collaborative filtering and even nearest neighbour search can help (given you have created good embeddings for the products using neural networks with multimodal input).
You would initially want to sort the list by frequency then date. Once, you have that you need to find related items to ...
This is often called the cold start problem.
There are many options to initialize:
Domain expert suggestions
Most popular suggestions from another platform
I got an answer to this same question in here.
Mainly, what is says is:
In general, softmax of catalog implies a fixed set of output items. Thus, whenever new items are added to the catalog, you'll have to change the output layer and retrain the model. In addition, training with a large softmax layer is time-consuming. Typically, the softmax layer is ...
The biggest challenge is probably how to measure the performance of your model. binary classification you can use Accuracy or AUC for example - but in multi-class it would be harder.
Measuring error in Recommendation systems is tricky in general. Different from typical classification problems. Predicting an absolutely amazing item to be shitty has a ...