Neuroscience is still trying to "find" how the mind (and language) somehow "works". Is there any theory linking a (low-dimensionality) embedding space (like word2Vec) to a mind (linguistic) model? Any Cognitive Linguistics theory?
$\begingroup$
$\endgroup$
4
-
$\begingroup$ My guess is probably not: word embeddings are used mostly because they often work better than the previous alternative, not because they correspond better to the human mind. $\endgroup$– ErwanApr 28, 2021 at 14:10
-
$\begingroup$ Thanks. Even if they dont correspond to human mind... is there a better way to represent the human mind in a way that a computer can understand? $\endgroup$– Raul AlvarezMay 1, 2021 at 9:05
-
$\begingroup$ I'm not aware of anything better in this particular way, at least not in mainstream NLP. There is a part of the general computational linguistics community which focuses on formally representing language as some kind of mathematical model. These works are to some extent related to linguistic and cognitive observations, but afaik not at the level of neurons, it's very far away from DL research. See formal semantics, DRT, ESSLLI21. $\endgroup$– ErwanMay 1, 2021 at 10:00
-
$\begingroup$ Thanks for this. Below I found one paper linking word vectors to (somehow) areas of the brain "activated" by words/concepts. $\endgroup$– Raul AlvarezMay 1, 2021 at 13:40
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Some initial steps where taken here:
Connecting concepts in the brain by mapping cortical representations of semantic relations
To represent words as vectors, we used a pretrained word2vec model21. Briefly, this model was a shallow neural network trained to predict the neighboring words of every word in the Google News dataset, including about 100 billion words (https://code.google.com/archive/p/word2vec/). After training, the model was able to convert any English word to a vector embedded in a 300-dimensional semantic space (extracted through software package Gensim56 in python). Note that the basis functions learned with word2vec should not be interpreted individually, but collectively as a space.
Applying the encoding model to the differential vector of a word pair could effectively generate the cortical representation of the corresponding word relation. With this notion, we used the encoding model to predict the cortical representations of semantic relations. For each class of semantic relation, we calculated the relation vector of every word pair in that class, projected the relation vector onto the cortex using the encoding model, and averaged the projected patterns across word-pair samples in the class.