1
$\begingroup$

I'm trying to implement a product categorization ML model on a dataset with the following structure: Data sample

I want to my model to be able to predict the correct category that the product should fall under, based on product description and name.

However, I will be implementing this together with a GUI which allows some user input.

For example, a new product name with a description gets added to the table: New entry before feedback training

The user will be presented with the following options (completely made these up) and has to select one:

Kitchen furniture - 65%

Home decorations - 29%

Kitchen Appliances - 6%

User will click on 'Home decorations'. This gets fed back to the model. Next time the model encounters something similar, such as: New entry after feedback training

The user will be presented with more accurate predictions, where this time they have the same options to choose from, but with different predicted accuracy:

Home decorations - 70%

Kitchen furniture - 20%

Kitchen appliances - 10%

Therefore, the model has learned from that feedback and has become more accurate. I've done some research around this and it has pointed towards Reinforcement Learning. However, I couldn't find anything too similar and I am not THAT skilled in ML, so please point me in the right direction in terms of what Python libraries to use, what ML models to look at and maybe even previous implementations.

Thanks!

$\endgroup$

1 Answer 1

1
$\begingroup$

This could be framed, as a first approximation, as a supervised learning classifier, where, based on the input texts (both name and description), you can build a series of features to build your classification model.

One option is:

  • tokenize (split into words) your texts (both name and descriptions)
  • filter some not useful (presumably) words like preprositions and other so called stop words (look at libraries like nltk for language processing
  • select the most frequent words of your bag-of-words based on all of the categories you have until now; this is something you can find out by looking at a frequency bar plot for your entire words dataset
  • find the frequency of ocurrence of each word in each of your name-description sample, where each row of your dataset could be something like:
kitchen bathroom storage cooking microwave oven ... CATEGORY_label
0 0 0 0 2 1 ... 1
1 0 1 0 0 0 ... 3
...

where label 1 is your kitchen appliances category and so on...

This would end up in a multi-class classifier, since you are trying to classify among several possible categories.

As many new entries as you end having, more key words you will have for each category.

This is the easiest approach (based on words counting) you could begin with, since, for natural language processing, you can go on lagter with other approaches: TF-IDF instead of just counting words, and other more sophisticated like word embeddings

$\endgroup$
4
  • $\begingroup$ Thanks for that! Would this approach enable user input which then gets fed back into the model to help improve model performance over time? This is a user requirement. $\endgroup$ Commented Oct 6, 2021 at 14:01
  • $\begingroup$ if I understood your use case, you want to provide a level of confidence to the end user for selecting the right category based on the entered name and description (so, this is the model inference); once the user selects the category (the actual one), this means a new entry for your history training set, used in a next model training phase $\endgroup$
    – German C M
    Commented Oct 6, 2021 at 14:19
  • 1
    $\begingroup$ That's spot on! Thanks for the clarification $\endgroup$ Commented Oct 6, 2021 at 14:38
  • $\begingroup$ please do not forget to validate the answer if you are happy with it ;) $\endgroup$
    – German C M
    Commented Oct 6, 2021 at 15:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.