I would like to achieve a classification of a text input into predefined categories. From what I have understand unsupervised approach are unfeasible if my target label is something very rare in pretrained models (I have labels about specific industrial processes). Is this true?

Otherwise I could try an approach in which I label for example 1000 input texts using all the different labels and use a supervised approach with very few labeled data. Should this help someway the learning process? And what methods could I use in this case?


1 Answer 1


A feasible approach would be to take a pre-trained model, like BERT, and fine-tune it on a small labeled dataset.

For that, you may use Huggingface's Transformers, which makes all the steps in the process relatively easy (see their tutorial on doing exactly that: https://huggingface.co/docs/transformers/training)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.