2
$\begingroup$

Apologies if this is naive, I am fairly new to the domain. I have a requirement where I am trying to classify 2 types of text data, i.e, I have got 2 classes to classify my data upon. I am able to get acceptable results for them using word vectors, dimensionality reduction and then putting the data through to LinearSVC for classification. But my model is biased towards sentences with longer lengths. I know tfidf can help in this, but is there a way to apply that with word vectors, I dont want to lose its ability to predict on unseen similar data?

A follow up problem to this is, when there is data from some other class apart from the 2 classes that I have trained my data upon. I would like my model to be able to predict that the data doesn't belong to any of the 2 classes. Currently, it just predicts 1 of the 2 classes randomly.

I was thinking if I should first put my data through topic modelling, which would give an idea on what topic the text data is. Based on keywords from topic modelling, we can detect if my text data belongs to one of the 2 classes or not and then put it to my classification model for final prediction. But this doesn't seems very clean and I can see it failing as there's too much dependency on keywords generated by topic modelling. Is there any other finer and better way to do this?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

This is called an open-class text classification problem, it's used in particular for some author identification problems. I don't have any recent pointers but from a quick search I found this article: https://www.aclweb.org/anthology/N16-1061.pdf

In the field of author classification there is a similar problem called author verification, which can be treated as a one-class classification problem. You could consider using it in this way:

  1. one-class classification between "known classes" vs. others
  2. regular classification between the known classes
Improve this answer
$\endgroup$
6
  • $\begingroup$ Thanks alot. That worked for me! $\endgroup$
    – Avneet Singh
    Commented Jun 26, 2020 at 20:48
  • $\begingroup$ Can you please answer the first bit as well. How to neutralize bias in my model caused from length of input. Is it possible to somehow use tfidf alongwith word vectors ? $\endgroup$
    – Avneet Singh
    Commented Jun 26, 2020 at 20:51
  • $\begingroup$ @AvneetSingh you didn't give any detail about how your vectors are built: one hot encoding? word frequency? Did you try any kind of normalization? You could indeed use TFIDF weights, but it's unlikely to help with the sentence length bias (there's no particular issue using TFIDF with unseen data btw). In general the problem of length bias is hard to completely solve, but sometimes it can be improved. $\endgroup$
    – Erwan
    Commented Jun 27, 2020 at 11:16
  • $\begingroup$ Thank you so much for getting back. I have used spacy vectors to create vectors with 300 dimensions. I haven't tried any kind of normalization. My understanding is words in sentence are represented by numerical value in vectors, if I normalize vectors, that would end up changing the numerical values of vectors (and hence words too), isnt it? $\endgroup$
    – Avneet Singh
    Commented Jun 27, 2020 at 18:21
  • $\begingroup$ @AvneetSingh it sounds like you're using words embeddings as vectors right? if yes this is not compatible with TFIDF or normalization indeed. In traditional vectors the values are usually the word frequency, so it makes sense to normalize or use weights. I'm not very skilled about word embeddings so I suggest you ask another question about this, it's better to focus on one point. $\endgroup$
    – Erwan
    Commented Jun 27, 2020 at 19:30
0
$\begingroup$

Apart from your desired two classes, relabel all other classes as a third class and then train your model on a three class classification problem.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you, but that would just mean that I need to know what I am expecting in all the other cases. Which essentially is the problem. We don't know what data there can be in all the other cases. $\endgroup$
    – Avneet Singh
    Commented Jun 23, 2020 at 18:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.