So in the paper for OCR pr LaTex formula extraction from image What You Get Is What You See: A Visual Markup Decompiler, they pass the features of the CNN into RNN Encoder. But there is problem that rather than passing the features directly, they have proposed a solution to change it into the grid.
Extract the features from the CNN and then arrange those extracted features in a grid to pass into an RNN encoder. This is the exact language they have used.
What is meant by that? Theoratically speaking, if I have an
CNN without any Dense/Fully Connected layer and produces an output of
[batch,m*n*C], then how can I change it in the form of a
grid?? Please see the picture below. So after getting the output from the
CNN, they have chnged it somehow before passing it to
RNN. What is the method that one can use to get this transformation?
So if I have to pass something to
keras.layers.RNN()(that_desired_grid_format), what should be this grid format and how can I change it?