is it possible to enforce image-to-text models like ViT or a simple CNN+Transformer to achieve character-wise accuracy?

Here's the context of my project: I am developing a model to extract some targeted phrases and numbers from individual pdf pages and I would like the numbers, especially, to be character wise accurate.

Correct me if I am wrong, but ViT or other CNN+Transformer models make context-based estimations which is not the same as character-wise estimation, am I right?

One idea that I have is to tokenize the ground truth string on a character level instead of word but I am not sure how feasible this is.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.