My question might sound a little bit stupid but I am trying to come up with a way to measure the relevance of the title of a text, let's say a piece of news headline, to the content. My idea would be to work with word2vec and have a go with cosine similarity. Are there any more efficient and proper ways to work with this task?
Word2Vec and cosine similarity are certainly a plausible solution. Note that the longer the text is, the more function words are in the text, therefore the average Word2Vec vectors are more similar to each other with the growing text length. This can be partially solved by removing the stop-words.
Another method to try might be cosine similarity with representation from contextual embeddings like ELMo or BERT. Also if you have some training examples (small hundreds might be enough) training a classifier based on BERT should work.
Finally, the fanciest solution is using the summarization model. Generating a title from an article content is one of the tasks that is done in summarization literature. If your data is similar to datasets that are used for training summarization models (or you have enough domain-specific data to train your own summarization model), you can use the model to estimate the probability of the title given the content. Even models that perform poorly on generation, perform well when used discriminatively.