Many questions on a single post, let's go step by step:
Assume you have taken the average of all vectors of words referring to animals (like family and species names, common terms like "cat").
Would one say this averaged vector represents the "prototypical animal"?
For a given dataset (of documents) you can refer to this vector as the average of animal vectors. If this vector can be understood as prototypical animal depends on the dataset, the dimensions chosen for the embedding, the type of embedding, the vectors chosen for making the average, etc.
For example if you are training an embedding with two dimensions, and your corpus (set of documents) consists of two classes of documents (one dealing with football terms and the other with animal terms), it is quite probable that one of the dimensions of the resultant embedding vectors will refer to animal terms.
Would one expect this vector to be similar to the vectors of "animal" and "animal-like"?
Again it depends on the dataset, dimensions chosen, vectors chosen, weights for those vectors while making the average, etc. You are not guaranteed a 100% coincidence.
How would the averaged vector probably look like? Will for example many dimensions be cancelled out by averaging, and only a small number of non-vanishing dimensions remain?
Let's assume your corpus is only dealing with animal terms (e.g. veterinary documents), let's assume you chose 4 dimensions for the embedding space: in order to have cancellation of dimensions (i guess you mean average is 0
) your embedding training must have discovered vectors with opposite values for some dimensions, e.g. it may have learned that mammals are the opposite of reptiles (or directly non-mammalian), then assume you have those two vectors normalized and summed, the value for one of the dimensions may cancel; again it doesn't have to be exact.
Will this characteristic "spectrum" be said to represent "animal-likeness"? Or won't there be something characteristic to be found?
With embedding vectors you can find similar vectors, so yes you can check for animal-likeness for each term, again it depend on your dataset; what if the word four
appears always near the word dog
? Clearly the training process will assume that four
has some animal-likeness but we know the word as such is not related to any animal, plus you can have two legs animals (e.g. birds).
I'd recommend you this post towardsdatascience: The Simple Approach to Word Embedding for Natural Language Processing using Python, it use a custom corpus and obtains the corresponding embedding set. Remember those are not contextual embedding vectors (like those used in Transformers) so each word can have at most an embedding vector.
To clarify the difference between word embeddings and contextual embeddings: stackoverflow: differences between contextual embedding and word embedding