Improve text classification accuracy

Question

Task:

I am building a text classification for salary prediction for data science jobs. I want to achieve at least 70 percent accuracy.

Data:

• Features: Consists of job descriptions of data science, data engineering, data analyst jobs of about 1800 samples
• Target: Target is the salary column binned into 5 different categories. I obtained this using the pandas q cut. The salary is heavily right skewed.

Problem:

The problem is that the models I train are always in range of 40 to 50 percent accuracy. I have tried different models like Random forests, SVM, Logistic Regression with Bag of words model and Tf-idf. But accuracy doesn't increase?

Questions:

• Should I be trying different word embeddings like glove or word 2 vec?.
• Should I try better models or move on to try neural networks?
• Is the bad accuracy because the text data and my target is completely uncorrelated?.
• Should I try to different binning lengths for target and try different number of bins?
• The data also has some samples which unrelated to data science but have software engineering in their titles, should I remove those?

[

Answer 1

You can try using simple transformer architecture, you can find reference for tuning it on hugging face. I think it will be able to get more context from the data you described. Here is a useful link:https://huggingface.co/docs/transformers/v4.15.0/training

Answer 2

I've recently finished a similar project using this open-sourced model which is based on the DeBERTaV3-base. https://huggingface.co/knowledgator/comprehend_it-base

It performs well in the zero-shot and few-shot learning settings. There is a framework for fine-tuning, which makes this model efficient in training using nearly 8 training examples/label. https://github.com/Knowledgator/LiqFit