I have a decent understanding of word embeddings (at its core, one can think of a word being converted into a vector of, say, 100 dimensions, and each dimension given a particular value... this allows to do math with the words, also it makes the training sets to be non-sparse...)
But today something came to my mind, what about punctuation symbols such as , . () ? ! ... ?
They do have a huge impact on the meaning of sentences and, like words, the position and context in which they are used is relevant.
So the question is, how should this be modeled? are pretrained sets like
GloVe including punctuation symbols? should I simply remove punctuation symbols from text?