I'm trying to map similar ngrams using Wordnet and synsets. For example:
elder brother and
older sibling should map to the same entity.
What would be the best way to implement this? I've been thinking and so far I've only come up with a brute-force approach checking for each synset of each word and trying to find a similar word or else add them as a new entity.
I am wondering whether there are any better methods of implementing this?
from nltk.corpus import wordnet as wn from nltk.stem import WordNetLemmatizer l = WordNetLemmatizer() older = 'older' elder = 'elder' older_lemma = l.lemmatize(older, pos=wn.ADJ) elder_lemma = l.lemmatize(elder, pos=wn.ADJ) for syn in wn.synsets(older_lemma): if elder_lemma in syn.lemma_names(): print(syn)
Ideally, I would want to extend it to n-grams and I'm searching for a better way to do this.
Edit : I'm not looking for vector based solutions.
What I'm thinking of is some sort of crude but fast similarity algorithm that can give me a crude representation of how close can two words/synsets possibly be. That way I could eliminate most of the absolutely dissimilar words saving time. I'm not sure if it exists