Word2Vec word vectors were being used for classification purposes in the early days. We could take word vectors of all the words in the sentence to get the resultant vector which could give us some sense of what this sentence would mean but this was not enough for more complex scenarios where we need to extract more information from the sentence. for e.g. what this sentence is referring to etc.
Then came LSTM which tries to learn the information by using the relative ordering of the words in the sentence. It tries to learn what to hold after processing a few words(limitation of the architecture) in the order and using that information along with the next word vector to output the next information to hold. So it is always the interaction of a word with the overall information carried up to that time and the actual info for each word gets diluted as more words are processed.
To deal with this issue, BERT calculates dot product(simplifying a bit) for each word with the other words in the sentence(aka attention matrix) thus capturing every information it can. The dot product for each pair of words does not necessarily make sense but more or less they represent how much they 'attend' to each other. While training we usually fine-tune the attention matrix to give the right attention values in order to have correct predictions.
This idea is not enough because it again misses the relative ordering information for the words. So they added positional encodings to each word based on where it lies in the sentence. for eg. 1st occurrence of 'to' will have some x positional encoding added while 2nd occurrence of 'to' will have y position encoding added to its vector, both will have same vector in the starting.
This idea also made better predictions for classification tasks as well as for question answering and other complicated tasks which were not possible before BERT.