Topic classification on text data with no/few labels

Question

I would like to achieve a classification of a text input into predefined categories. From what I have understand unsupervised approach are unfeasible if my target label is something very rare in pretrained models (I have labels about specific industrial processes). Is this true?

Otherwise I could try an approach in which I label for example 1000 input texts using all the different labels and use a supervised approach with very few labeled data. Should this help someway the learning process? And what methods could I use in this case?

Answer 1

A feasible approach would be to take a pre-trained model, like BERT, and fine-tune it on a small labeled dataset.

For that, you may use Huggingface's Transformers, which makes all the steps in the process relatively easy (see their tutorial on doing exactly that: https://huggingface.co/docs/transformers/training)