I am training a classifier which is supposed to take the name of a product as input. For this purpose I want to finetune a pre-existing fasttext model on my article names.
My code looks like this
import fasttext
# Load the pre-trained model
pretrained_model_path = 'models/cc.de.300.bin'
ft_model = fasttext.load_model(pretrained_model_path)
# Fine-tune the model on additional data
data_path = 'data.txt'
ft_model_finetuned = fasttext.train_unsupervised(data_path, pretrained_vectors=pretrained_model_path, epoch=10, lr=0.1)
This yields me:
Warning : `load_model` does not return WordVectorModel or SupervisedModel any more, but a `FastText` object which is very similar.
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In [1], line 9
7 # Fine-tune the model on additional data
8 data_path = 'data.txt'
----> 9 ft_model_finetuned = fasttext.train_unsupervised(data_path, pretrained_vectors=pretrained_model_path, epoch=5, lr=0.01, dim=300)
File ~\anaconda3\envs\bachelorarbeit\lib\site-packages\fasttext\FastText.py:559, in train_unsupervised(*kargs, **kwargs)
557 a = _build_args(args, manually_set_args)
558 ft = _FastText(args=a)
--> 559 fasttext.train(ft.f, a)
560 ft.set_args(ft.f.getArgs())
561 return ft
ValueError: Dimension of pretrained vectors (-5924836670674077010) does not match dimension (300)!
but
ft_model.get_dimensions()
returns 300 as it should.
this works fine as well...
fasttext.train_unsupervised(data_path)
Read 1M words
Number of words: 20872
Number of labels: 0
Progress: 100.0% words/sec/thread: 20555 lr: 0.000000 avg.loss: 1.450562 ETA: 0h 0m 0s
So I can use the pre-trained model, it tells me it has 300 dimensions when I check for it in my code. I can train a model on my data. What I cant do is fine tune the model.
The training data follows this schema btw: article name 1\n article name 2\n
So for example:
blueberry muffin
chocolate chip cookie
outdoor flashlight
