0
$\begingroup$

I am currently working on a project that needs product range analysis. It's an ecommerce dataset that has 7 columns: InvoiceNUm, StockNum, Description, Quantity, InvoiceDate, UnitPrice and CustomerID. This means I have to create a category for all the products and I can only create category names based on the description column.

Below is a sample list of description and the category that I need to create.

enter image description here

How do I approach the task of creating category for each product description? Like which model suits this problem? Any suggestions highly appreciated. Thanks in advance.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
0
$\begingroup$

I think a first approach would be making a counting matrix, and assess which words area more representative and map the most important ones, for example, in the sample that you've showed "garden" would have a big volume in this matrix, so assign the a simple rule such as "has garden in description is fro garden category"

Improve this answer
$\endgroup$
2
  • $\begingroup$ That was also my first idea, I tried putting some categories based on the nouns in the description then will assign rule probably through a loop. However some descriptions can be confusing like for example MAGIC GARDEN FELT GARLAND can't be assigned to GARDEN but to ARTS, CRAFTS & GIFTS. And I have 540,000 records. $\endgroup$
    – SamanthaK
    Jul 20, 2022 at 12:34
  • $\begingroup$ You could use a soft clustering in the flavor of a Gaussian Mixture, you can assign a probability of belonging a class instead of a hard belonging . $\endgroup$
    – Allan
    Jul 20, 2022 at 13:22
0
$\begingroup$

A starting point can be using BertTopic to cluster the descriptions into topics and then use the topic names as categories. Note that the category names you will get will be just short phrases extracted from your descriptions.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you. I'll check on BertTopic. I haven't learned that yet. So far I have only tried WordNetLemmatizer and word-tokenize then Counter but it didn't work. $\endgroup$
    – SamanthaK
    Jul 20, 2022 at 12:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.