9
$\begingroup$

Task:

I have a dataset with job titles and descriptions. The task is to predict tags for job by job title and description.

There are several tags for each job posting. Therefore, the number of labels for the model will be measured in tens of thousands.

Number of job postings = 78042

Number of unique classes (tags) = 1369

enter image description here

enter image description here

Questions:

Could you advise working types of neural networks (desirable in Keras)?

Or maybe you know how to solve this problem with the help of classical machine learning algorithms?

Still I will be very grateful for links to articles where similar problems are solved.

Improve this question
$\endgroup$
4
  • $\begingroup$ Please add more data-points about the task. How many labelled examples/inputs do you have? When you say thousand output classes, is it lower end (1000 classes) or 99999 classes? $\endgroup$
    – hssay
    Commented Aug 21, 2018 at 11:54
  • $\begingroup$ @hssay, updated $\endgroup$
    – lemon
    Commented Aug 21, 2018 at 12:33
  • $\begingroup$ Acording to your histogram you have a dataset that is already labeled with tags. So this is a supervised learning problem? In other words, you want to assign tags to new untagged job postings on the basis of the existing dataset? How many tags per posting? One? 3 or 4 (the median)? Up to 15 if necessary? Up to 43? $\endgroup$
    – knb
    Commented Aug 22, 2018 at 7:11
  • $\begingroup$ @knb, Yes, it is a supervised learning. Median value for tags number per posting is 4. $\endgroup$
    – lemon
    Commented Aug 22, 2018 at 7:25

3 Answers 3

Reset to default
7
$\begingroup$

The ratio of number of examples to number of classes is not large. There are few classes which have high number of occurrences (from the second graph) and the distribution seems to be following power law.

In such cases, I'll advise the following strategy,

  1. Sort your tags by number of occurrences and discard tags which occur very few number of times. This will make the problem more tractable.
  2. You can benchmark the accuracy that you get from classical machine learning techniques. Many classical methods support multilabel output, you can check the documentation of support in scikit-learn library here.
  3. You can do a mix of unsupervised learning and nearest neighbours approach. For example, learn doc2vec embeddings on your data including tags and suggest tags from nearest matching document for a new input. The number of documents is small by doc2vec standards and you will need careful tuning of doc2vec parameters.
  4. With neural networks, you can use more suitable loss functions like multilabel soft margin loss.

I'd advise you start with classical techniques to first benchmark the potential accuracy.

Improve this answer
$\endgroup$
2
$\begingroup$

Actually, you can solve these types of problems easily with deep learning. For a moment think of a chatbot which can generate answers given questions. If we think as you mentioned each time final softmax layer should predict a probability distribution which is similar to vocabulary size. But this isn't the case. We use a loss function called Noise Contrastive Estimatimation (NCE_LOSS). Here we sample most likely words and use them to compute the softmax layer. Here I will mention Tensorflow like to understand this scenario.

Improve this answer
$\endgroup$
2
  • $\begingroup$ could pls share a link with the practical implementation of this algorithm (maybe, github or ipython notebook)? I just do not quite understand how to implement it. $\endgroup$
    – lemon
    Commented Aug 21, 2018 at 12:36
  • $\begingroup$ There are many. Here is one - github.com/google/seq2seq $\endgroup$
    – Shamane Siriwardhana
    Commented Aug 21, 2018 at 16:57
2
$\begingroup$

You could follow along how instructor Rick Scavetta processes the "Reuters newswire dataset" of short news-items into 46 news-categories (e.g. Sports-baseball). Politics/USA). I think this is very similar to your problem.

R Markdown File

  • The video tutorials are currently in the making. I like the style.
  • Using deep learning and the R interface to Keras, he assigns only 1 class per newsitem, not 4 or more
  • Test Accuracy is 77% (but maybe you can do a better with a better lexikon)
  • To my knowledge, you need a lexicon with a ranking of words (how important they are. stopwords such as "the" and super-rare words need to be ranked low)
Improve this answer
$\endgroup$
1
  • $\begingroup$ Does anyone know if a similar example in Python? This is exactly what I'm looking for, but I don't know R. $\endgroup$
    – P A N
    Commented Sep 2, 2019 at 10:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.