Evaluation is based on the task, not the type of model used for it. In the tutorial that you link the task would be simple document similarity. Afaik a more common variant is the information retrieval setting, where the goal is to rank documents according to their similarity against a query document.
The tutorial mentions that they use a test set for which a link to a paper is provided. This paper explains that human annotators rated the similarity level between pairs of documents. That would be the standard way to obtain annotated data, but of course it takes a lot of manpower. It doesn't seem that the ground truth information is provided with the test set used in the tutorial though, I assume that this is why the author only proposes a manual inspection of the results.
In general there are probably many benchmark datasets (with ground truth annotations) publicly available for similar tasks, but they are not always easy to find and I'm not very knowledgeable about these. I think a good place to start would be the SemEval series of competitions, every year they release various datasets related to this kind of task.