2
$\begingroup$

Task:

I am building a text classification for salary prediction for data science jobs. I want to achieve at least 70 percent accuracy.

Data:

  • Features: Consists of job descriptions of data science, data engineering, data analyst jobs of about 1800 samples
  • Target: Target is the salary column binned into 5 different categories. I obtained this using the pandas q cut. The salary is heavily right skewed.

Problem:

The problem is that the models I train are always in range of 40 to 50 percent accuracy. I have tried different models like Random forests, SVM, Logistic Regression with Bag of words model and Tf-idf. But accuracy doesn't increase?

Questions:

  • Should I be trying different word embeddings like glove or word 2 vec?.
  • Should I try better models or move on to try neural networks?
  • Is the bad accuracy because the text data and my target is completely uncorrelated?.
  • Should I try to different binning lengths for target and try different number of bins?
  • The data also has some samples which unrelated to data science but have software engineering in their titles, should I remove those?

Test data confusion matrix

[ salary bin distribution 2

$\endgroup$

2 Answers 2

2
$\begingroup$

You can try using simple transformer architecture, you can find reference for tuning it on hugging face. I think it will be able to get more context from the data you described. Here is a useful link:https://huggingface.co/docs/transformers/v4.15.0/training

$\endgroup$
1
  • $\begingroup$ I will try a transformer model but I think I have too less samples. Idk how well they will perform. $\endgroup$
    – Sendhan
    Feb 1 at 9:49
2
$\begingroup$

I've recently finished a similar project using this open-sourced model which is based on the DeBERTaV3-base. https://huggingface.co/knowledgator/comprehend_it-base

It performs well in the zero-shot and few-shot learning settings. There is a framework for fine-tuning, which makes this model efficient in training using nearly 8 training examples/label. https://github.com/Knowledgator/LiqFit

$\endgroup$
1
  • $\begingroup$ Hello, do you have a github for this project? Would be helpful to learn a lot. $\endgroup$
    – Sendhan
    Feb 1 at 12:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.