How is the classification step performed in Swin Transformer?

Question

I've read the original paper of the Swin Transformer, and understand it quite well apart from the last step: the classification. In the paper, this is completely brushed over, and none of the blog posts talking about this architecture adress it as well.

In practice, how is this done? The transformer outputs a series of vectors (an array), so we can't just apply an FC layer to obtain a vector of probabilities. Are the different output vectors concatenated together (flattening) and fed to an FC, or are they fed to a decoder as in the standard transformer architecture?

Answer 1

There are many ways of doing this, and the authors did not specify it on purpose. One can aggregate the output vectors in various ways, or just choose the first position (like in BERT) solely to be passed on the the MLP.