I'm trying to build a model that maps images of math formulas into LaTeX markup.

I found an acticle (https://arxiv.org/ftp/arxiv/papers/1908/1908.11415.pdf) that proposes an encoder-decoder architecture that solves this problem.

In encoder after the work of CNN it's proposed (page 3) to add following signals to generated feature maps "in order to retain the spatial locality information":

enter image description here

Here the authors refer to positional encoding part of Transformer model "tailoring the 1-D positional encoding technique to 2-D".

In original Transformer model (from article "Attention Is All You Need") the position encoding is calculated like this:

enter image description here

Where "pos" is just an index number of word in given sentence (correct me if I'm wrong).

And here comes the question: what is "x" and "y" in first formula?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.