What is the best way to regularize latent embeddings, I have two solution in my mind but I'm not sure which one to use over other.

  • In batch-wise training regularize over the whole embedding matrix, but this would be too expensive computation wise

  • Regularize over the item embeddings in the current batch but as real world datasets tends to have non-uniform distribution of items in the dataset, perhaps model will penalize commonly occurring items more over the long tail items

Is there any better way to deal with such cases or any improvisation over the above methods?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.