0
$\begingroup$

https://github.com/google-research/bert/blob/master/modeling.py#L491-L520

The code of BERT is one of the implementation. But it is not what I need.

I search a lot but can not judge.

But where is the code which seems uses cos and sin?

| improve this question | | | | |
$\endgroup$
0
$\begingroup$

Here is the code implemented by TensorFlow:

Implementation 1:

position embedding position embedding

Implementation 2:

position embedding

| improve this answer | | | | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.