2
$\begingroup$

I have a multiclass classification problem where, in the test set, there is only one entry for each possible class. In my particular problem we want to guess the author of a text, and we have 20 different authors. The test set contains 20 texts, one of each author.

I have many texts of each authors in the training data. I can't change the content of the test set, it's these specific texts that I need to classify.

  • Does this kind of task has a name, so I can google previous work on it more easily ? I'm not talking about author identification specifically, but having 1 entry for each class.

  • Do you have any suggestion of algorithm that would work well for this task ? Maybe something that gives ranked results or a confidence value ?

Thank you

$\endgroup$
3
  • 1
    $\begingroup$ Slightly unclear what your train set is then, but if you want to maximize the (log of) the product of probabilities, it is the maximum matching problem. $\endgroup$
    – Valentas
    Jan 30, 2020 at 11:59
  • $\begingroup$ Isn't it possible for you to partition your train data if that has good enough volume out there, and remove target variable from it before training? Agree with @Valentas as to how different is your train set then! $\endgroup$ Jan 30, 2020 at 12:51
  • $\begingroup$ I edited to add informations about training data. I have all the training data I need, but I can't change the test set. It's these specific texts that I need to find the author of. $\endgroup$
    – Alpacaman
    Jan 30, 2020 at 14:03

2 Answers 2

1
$\begingroup$

With a single exemplar per class, there is a limit to appropriate evaluation metrics. There can be no estimate in the variability of performance.

It is best practice to look at test dataset only once so the current setup will have limited value.

A related machine learning field is one-shot learning.

It might be better to reframe the problem as a case study since there is not enough data for effective machine learning.

$\endgroup$
0
$\begingroup$

When talking about the test data, we can have any number of examples of each class for inference. It doesn't matter if it has 20 classes with one example each.

You can use a CNN classifier for this type of a problem but if one book can be written by more than one author you should use sigmoid activation in the last layer rather than using softmax.

You can see this article which address an usecase similar to yours - Link

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.