0
$\begingroup$

I have a dataset of machine-generated sequences that are not natural language, but the order of the words in the sequence is important. I want to create word embeddings using BERT to capture the sequential relationships between these words.

NOTE: The vocabulary that I have in my data is not present in the pre-trained model. My Question is - Do I have to build the entire model from the scratch or Can I somehow change the vocabulary and use pre-trained BERT to create word embeddings?

Example of my Data *(list of sentences)* = [‘ixeg6164 ox78dsf12 lx3cd875’, ‘duish7 oiu587 kj854j 987hdk’ …]

Example Vocabulary = ["ixeg6164", "ox78dsf12", "lx3cd875"....]

Can anyone guide me how to achieve my goal ?

Improve this question
$\endgroup$

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.