Spatial positional encodings Vs Learned positional encodings(Object queries)

Question

I have been trying to understand facebook's Detection transformer(DeTr) paper.

Architecture

Most of the explanation about the architecture is straightforward.
I don't especially understand the concept of object queries.

Details

In the paper,

A transformer decoder then takes as input a small fixed number of learned positional embeddings, which we call object queries, and additionally attends to the encoder output.

There are 2 kinds of positional encodings in our model: spatial positional encodings(fixed) and output positional encodings(Object queries).

FYI:
Learned positional encodings and output positional encodings are same.

Questions

1. What are these "learned" positional encodings(object queries)?
2. how are they different from spatial positional encodings?
3. When are they exactly learned?

Answer 1

Transformers typically convert a sequence of vectors of a fixed length, d, to an output sequence of vectors of the same length.

How you arrive at the sequence of vectors to feed into the transformer depends on the problem you're trying to solve.

• For language translation , you embed your word into a d dimensional embedding.
• For image recognition you learn a mapping from fixed size patches of the image to a d dimensional vector.

But sometimes, you do not have an input vector to provide to the model. Example cases taken from the BERT include

• When the first output is a special case, the classification of the whole sequence. This is the first [CLS] token.
• When the input token is masked, it's replaced with a [MASK] token.
• When you want to show the separation between two sentences, you use a [SEP] token.

In case of the DeTr paper, the decoder stage does not have any inputs. So, they simply learn a d dimensional embedding at each position, and call these object queries, since these are used to query the learned encoding of the input image to generate the outputs.