1
$\begingroup$

I want to reduce the dimensionality of the BERT word embedding to, let's say, 50 dimensions. I am trying with PCA. I will use that for the document classification task.

Now for training PCA, should I train on the entire dataset by using all the word vectors from the entire data set at once that is:

pca.fit_transform([all_the_word_vectors_of_the_dataset])

or word vectors per document, that is:

for document in train_dataset:
    pca.fit_transform([word_vectors_of_current_document])
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.