I want to reduce the dimensionality of the BERT word embedding to, let's say, 50 dimensions. I am trying with PCA. I will use that for the document classification task.
Now for training PCA, should I train on the entire dataset by using all the word vectors from the entire data set at once that is:
or word vectors per document, that is:
for document in train_dataset: pca.fit_transform([word_vectors_of_current_document])