I am supposed to train a classifier with historical shopping data that predicts the probability of an item being returned. The only human language contained for each purchase is the name of the product. Since the purchases are from many different companies all over Europe, the product names are often in different languages.

What model would be best suited to encode these product names? Would you use a translator to translate these product names? They are often riddled with abbreviations or brand names and in my eyes would most likely not lend themselves well to translations.

(I am yet to receive the data set, so I dont know exactly how many different products exist and what languages are most common, however the data set is mostly likely dominantly german and english)


1 Answer 1


The main question is whether the product name helps predicting whether an item will be returned.... I don't see any clear case where the product name can give this kind of clue, so I simply wouldn't include the product name in the features. But I would probably still include this variable as a one-hot encoded variable, because it's likely that some products are returned more often than others.

  • $\begingroup$ I mean we could make the assumption that mainstream or branded names wouldn't be returned, can't me? People are more likely to not return Heinz or Coke, compared to ketchup or cola that are off brand. Same could apply if it was an outlet clothing store. I think its a relationship the model might capture. $\endgroup$ Oct 31, 2022 at 7:21
  • $\begingroup$ @DarknessPlusPlus I didn't think about that, you might be right but we don't know if OP's data has the brand name in the product name. $\endgroup$
    – Erwan
    Oct 31, 2022 at 11:06
  • $\begingroup$ The dataset contains data from roughly 20.000 individual retailers and therefore products differ largely. Some sell CBD infused dog foods, other's maybe outdoor gear. I do have an ID that identifies each retailer individually, but no real way of making connections between the type of goods individual retailers sell besides the product names. So if retailer 1234 and retailer 4567 both sell dog food, my only way of telling would be by the product name. I might try encoding them as a categorical just to check, but one hot encoding is not possible, since we are talking 200k+ products. @Erwan $\endgroup$ Nov 4, 2022 at 9:23
  • $\begingroup$ But most likely I still need a language model to model my product names. Any suggestions? @DarknessPlusPlus $\endgroup$ Nov 4, 2022 at 9:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.