0
$\begingroup$

As someone who is new to deep learning, I am only familiar with self-attention.

I'm designing a model. Imagine there are n data, which the $i_{th}$ data can be represented as a vector $x_i$. And the data has an attribute $a$, the attribute of the $i_{th}$ data is $a_i$. now I only use one query vector $q$ to multiply each data to calculate the result as the weight $w_i$ ($w_i=q^t * x_i$) , and use the weight of each data to multiply each data's attribute $a_i$ as the final result. ($A=\sum w_i * a_i$) Does this kind of attention exist? if so what its name is?

For example, $x_i$ is a 768-dimensional vector representing each sentence, and $s_i$ is a 3-dimensional vector representing each sentence's sentiment. $q$ is a 768-dimensional vector multiplied with each $x_i$ to produce each sentence's weight $w_i$. And the weighted sum $S=\sum w_i∗s_i$ is the overall sentiment of all sentences.

$\endgroup$
2
  • $\begingroup$ probably everything is being tested at some point related to attention, and if it's not famous, probably didn't worked out... however, it's not clear what you are trying to explain... try being more "technical", 1 graph/formula is worth more than 1000 words $\endgroup$
    – anon
    Aug 1, 2022 at 21:22
  • $\begingroup$ @AlbertoSinigaglia For example, $x_i$ is a 768-dimensional vector representing each sentence, and $a_i$ is a 3-dimensional vector representing each sentence's sentiment. $q$ is a 768-dimensional vector multiplied with each $x_i$ to produce each sentence's weight $w_i$. And the weighted sum $A=\sum w_i * a_i$ is the overall sentiment of all sentences. $\endgroup$
    – user900476
    Aug 1, 2022 at 21:31

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.