In my project I follow the retrieval augmented generation (RAG) approach. I want to create embeddings for my own dataset and use it in combination with llama-2. In the dataset are german annual reports, 548 reports as pdf-files with about 300 sites per report. Next, I want to load the data in a vector store, but first I think I have to create the embeddings.
And now, there are serveral questions and I need some best-practice:
Do I have to train my own embeddings model or can I use models like word2vec of the gensim package or a pretrained model like BERT and take the hidden state?
Can I use any embeddings model? I think they train on a specific corpus and if my words aren't in the training corpus, I will get bad result or what do you think?