Python - How to use ML to assign best possible weights to final stage of a NLP product title shortening algorithm to pick most descriptive parts

Question

TLDR: How to use ML to automatically assign best possible weights so my algorithm knows how much to value each feature to make a decision based on training data?

Hi everyone!

MY PROJECT:

I am using Python to create a product title shortening algorithm that does something like the following:

INPUT: 3.5mm Jack Wireless Bluetooth-compatible 5.0 Receiver Adapter Wireless Aux Receiver Adapter For Headphone PC Music MP3

OUTPUT (60 chars max): 3.5mm Wireless Bluetooth 5.0 Adapter for Headphone

Except that the titles I am working with are in Portuguese. The program does this through a simple NLP algorithm that is based on grammatical segmentation of the title, meaning it essentially splits the title into noun chunks and then, if appropriate, intelligently merges these chunks together to produce new chunks.

After the title has been segmented, I then run it through a prioritization algorithm that is incomplete and, as of now, simply takes segment length and position into account to output the final shortened title, meaning longer segments that come first take priority. It might be relevant to emphasize that for 75% of the cases the non-ML algorithm that I have at the moment already picks the best choice but it doesn't use any of the many potentially relevant features I describe below and seems to me to still have a lot of room to improve.

MY PROBLEM:

I am extending this prioritization algorithm by adding more features to consider such as number of nouns/adjectives/connecting words/known words/unknown words, word frequency in large corpus of text such as Wikipedia, word frequency in all of the titles, word frequency in the product descriptions, etc. I also extracted every possible adjacent word combination to form millions of N-grams then kept only those that repeated 5 or more times to form a list of thousands of terms that range from 2-7 words in length that should, in theory, be more descriptive in nature due to their repetitiveness.

Now, these are WAY too many parameters for me to be able to decide how much each should be valued. This seems to me to be a perfect application for ML!

MY EXPERIENCE WITH ML:

How would I go about doing this? I have created a Neural Network from scratch many years ago following Andrew Ng's famous course but that is the extent of my experience with ML. I have also taken Kaggle's short introductory courses, also a while ago.

FINAL QUESTIONS:

1. What is the best algorithm/library for this particular problem? Ideally I would like something extremely powerful, even if it's not completely trivial to set up, that I could run in Python.

2. How many training samples would be enough?

3. Any other tips?

Thank you all very much!!!

Answer 1

A good option is to use a customized Bert library.

Therefore, you would need some custom tokenization to detect some key parterns such as "5.0" using a tokenizer like ByteLevelBPETokenizer.

from tokenizers import ByteLevelBPETokenizer

import os
    
tokenizer = ByteLevelBPETokenizer()

tokenizer.train(your training data)

os.mkdir('./your_tokenizer')

tokenizer.save_model('your_tokenizer')

Then, you could use some sequence classification to know at least the category of the sentence.

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("your_tokenizer")
model = AutoModelForSequenceClassification.from_pretrained("bert-base-cased")

Now you could need to finetune your model, but you can try without finetuning it:

classes = ["Headphone", "Keyboard"]

sequence_0 = "USB keyboard"
sequence_1 = "3.5 Headphone white"
sequence_2 = "Portuguese keyboard"

keyboard = tokenizer(sequence_0, sequence_2, return_tensors="pt")
headphone = tokenizer(sequence_0, sequence_1, return_tensors="pt")

keyboard_classification_logits = model(**keyboard).logits
headphone_classification_logits = model(**headphone).logits

keyboard_results = torch.softmax(keyboard_classification_logits, dim=1).tolist()[0]
headphone_results = torch.softmax(headphone_classification_logits, dim=1).tolist()[0]

# Should be keyboard
for i in range(len(classes)):
    print(f"{classes[i]}: {int(round(keyboard_results[i] * 100))}%")

# Should be a headphone
for i in range(len(classes)):
    print(f"{classes[i]}: {int(round(headphone_results[i] * 100))}%")

The code is inspired from this site: https://huggingface.co/transformers/task_summary.html

With enough data, you should be able to train your model with other languages. The amount depends on the variability of the possible names.

Note: you could try afterwards smaller models, as the words' structure is simpler than classic litterature.