0
$\begingroup$

Suppose if I have 2 sentences: "My name is Alex" "Alex is my name"

If I am using a RNN, After processing both the sentences, Will the final output vector be the same?

Because RNN basically shares the weights, And both the sentences have the same number of words,Shouldnt the final output after processing the last word in both sentences be the same ?

I am well aware that when processing each word in RNN, The next word will be based on the current and previous processed words. But what about the full processing of both these sentences with same words. Will they have same final output?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

No, they will not have the same final output.

Although the weights of the RNN are the same for each time step and the words are the same, their order is not and therefore the inputs and hidden states received at each time step will be different, and so will their outputs.

You said it yourself: The next word will be based on the current and previous processed words. . The next and previous words for each time step are not the same in two sentences with the same words but in different order.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Hey, I meant the final output after processing all the words. I am aware the intermediate outputs will be different. But I guess, A+B+C=C+B+A $\endgroup$
    – mewbie
    Jul 7, 2022 at 19:14
  • $\begingroup$ The mathematics is not as simple as an addition, I am aware of the tanh activations and the multiple combinations of weights and hidden states/ But the end result after all processing of the words should be the same right for both the sentence $\endgroup$
    – mewbie
    Jul 7, 2022 at 19:15
  • 1
    $\begingroup$ Yeah, my answer was indeed referring to the output after processing all words. The final output will be different. Precisely because the computations performed for each step are not as simple additions and are not commutative, the order in which they are applied defines the final result. $\endgroup$
    – noe
    Jul 7, 2022 at 20:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.