I don't have enough data (i.e. I don't have enough texts) --- have only around 4k words in my dictionary. I need to compare given words, then I need to representate it as embedding.
After the representation of words I want to clusterize it, find similar vectors (i.e. words). Maybe even then make a classification to a given classes (classification there unsupervised --- since I don't have labeled data to train on).
I know that almost any task can be solved "inside" BERT, i.e. using fine-tuning in final layer.
Since all described above, I have two QUESTIONS; answers/hints/anything really appreciated since i'm stuck on that:
- How to just extract embeddings from BERT using some dictionary of words and use word representations for futher work?
- Can we solve inside BERT using fine-tuning the next problem: a). Load dictionary of words into BERT b). Load given classes (words representing each class. E.g. "fashion", "nature"). c) Make an unsupervised classification task?