0
$\begingroup$

I'm training a Logistic Regression classifier on text data. I found that many of my data points have more than one target class. Is it possible to modify my model to output more than one class based on the data. I plan to split multi-class data points in my training set into distinct classes(i.e. if one x has 3 classes, I will split that text into three so that each different text has a unique class associated with it). Then when I predict on the test data if I will output n classes such that

Probability(Class_1)+Probabilty(Class_2)+...+Probabilty(Class_n)>0.95

I will use the prdict_proba method of LogisticRegression for this. Is it a correct way of doing this?. your help is much appreciated.

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
1
$\begingroup$

We can't use the same data with a different label in the same model.
It will confuse the model.
What you need is multiple models e.g. OneVsRest strategy. i.e. one for each Class.

Little clarification-
What you are looking for is Multi-label, not Multi-class. Please check the internet on this. May read this SO Answer for a quick understanding.


You can implement an OneVsRest multi-label yourself but better use the Scikit-learn implementation Link.

OneVsRestClassifier can also be used for multilabel classification. To use this feature, provide an indicator matrix for the target y when calling .fit.

What it means is to encode your data in a multi-label format when calling fit().
You may use the from sklearn.preprocessing import MultiLabelBinarizer for this.

classif = OneVsRestClassifier(LogisticRegression())
classif.fit(X, Y)

Read this SO Answer for an example.


You may also use a sklearn.multioutput import MultiOutputClassifier wrapper for the same. Check this SO Answer

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks, @10xAI for your answer. Could you please elaborate on the first sentence "We can't use the same data with a different label in the same model. It will confuse the model."? $\endgroup$
    – NAS_2339
    Commented May 25, 2021 at 4:55
  • $\begingroup$ I meant - Let's take an example, Data_1 = [1,2,3], y=CAT, Data_2 = [1,2,3], y=DOG. Same X but different Y. What should the model learn ? $\endgroup$
    – 10xAI
    Commented May 27, 2021 at 15:29
  • $\begingroup$ As far as I understand, in multi-class classification, we have n models corresponding to n class with a one-vs-rest approach. So for each x, we get n probabilities. We take the highest probability among them. In an animal classifier, there could be a case where p(Dog)>0.5 and P(Cat)>0.5, but the function returns Dog since p(Dog)>P(Cat). Can I tweak the model to output both Cat and Dog. Are there any pitfalls for this approach? $\endgroup$
    – NAS_2339
    Commented May 27, 2021 at 18:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.