I have a dataset of machine-generated sequences that are not natural language, but the order of the words in the sequence is important. I want to create word embeddings using BERT to capture the sequential relationships between these words.
NOTE: The vocabulary that I have in my data is not present in the pre-trained model. My Question is - Do I have to build the entire model from the scratch or Can I somehow change the vocabulary and use pre-trained BERT to create word embeddings?
Example of my Data *(list of sentences)* = [‘ixeg6164 ox78dsf12 lx3cd875’, ‘duish7 oiu587 kj854j 987hdk’ …] Example Vocabulary = ["ixeg6164", "ox78dsf12", "lx3cd875"....]
Can anyone guide me how to achieve my goal ?