From all the docs I read, people push this way and that way on how BERT uses or generates embedding. I GET that there is a key and a query and a value and those are all generated.
What I don't know is if the original embedding - the original thing you put into BERT - could or should be a vector. People wax poetic about how BERT or ALBERT can't be used for word to word comparisons, but nobody says explicitly what bert is consuming. Is it a vector? If so is it just a one-hot vector? Why is it not a GLoVE vector? (ignore the positional encoding discussion for now please)