Is there any way to modify word2vec or BERT to extend finding out embeddings for words that were not in the training data? My data is extremely domain-specific and I don't really expect pre-trained models to work very well. I also don't have access to huge amounts of this data so cannot train word2vec on my own. I was thinking something like a combination of word2vec and the PMI matrix (i.e. concatenation of the 2 vector representations). Would this work, would anyone have any other suggestions, please?
Thanks in advance!