Can BERT, GPT or other contextualised embedings be used for finding word definitions? What would be the most effective and not complicated approach for tackling a sample task as described below.

Map the meaning of the word 'bank' in the sentence "I was walking along the bank of the river" with one of the definitions listed in the WortNet database (or other word-sense lookup table).

  • $\begingroup$ For the record, the task of mapping a word usage in a sentence to a particular sense (among several possible senses) is called Word Sense Disambiguation. It's a very standard task, there's a lot of literature about it. $\endgroup$
    – Erwan
    Feb 6, 2020 at 23:38

1 Answer 1


BERT generates contextualized word embeddings, which means that BERTprovides the most accurate embeddings when a word is in a sentence(context). For each of the words within the sentence, BERT will generate a vector of numbers. In your case, you will have a good representation of the word "bank". So if you have a sentence for all the other words that you are trying to match with "bank", all you need to do is to extract the embeddings for them, then compare them with a similarity matric (i.e. cosine similarity). this way you are ranking them to see which ones are most correlated to "bank".


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.