1
$\begingroup$

Under what circumstances would BERT assign two occurrences of the same word similar embeddings? If those occurrences are contained within similar syntactic relations with their co-occurrents?

$\endgroup$
0
$\begingroup$

It's really hard to say if the BERT assigns similar embeddings to the same words in the similar context. Most likely it will not. This is because, the embeddings, is not just function of context (other words in the sentence), but the position as well. Hence, even if the words occurs in similar context, with different positions, they can have different embeddings.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.