0
$\begingroup$

we are trying to build an in house OCR system to extract alphanumeric strings from images (kindly note, most of our clients cant afford to send their data onto the cloud so it rules out any API from the googles and microsofts..and also tesseract is way too slow for our purpose ..we are talking 10's of users simultaneously uploading images .. tesseract, i guess was never designed to be used as such). We have already started using a CTC based approach to train on our data sets but there's a fundamental/conceptual issue. By definition, CTC finds a happy path to the most probable unit (in this case, a word) but alphanumeric strings by definition DO NOT have a pattern ! So i have a feeling CTC will not work for us. But juxtaposed to that is the multiclass classification approach (where we just predict every individual char) which has this big problem about how to split any image into specific characters before passing it to a CNN layer. We are a little clueless about the approach to be taken here. Any pointers will be heartily appreciated :)

$\endgroup$
1
$\begingroup$

CTC can also work with totally random text (and thus without any "word pattern").
I know that because I trained an OCR network with CTC loss on random text which was then able to read non random text (fortunately).
The CTC just sums the probabilities of the all the paths which are "compatible" with the target text (random or not).
The higher the sum of probas of "compatible" paths, the better the prediction.
If not done yet, you should read this very good article explaining the CTC loss.

Edit to clarify and answer questions asked in comments :

  • do u mean specifically alphanumeric strings ?
    Yes I mean random string with any character, alpha or numeric, it does not matter (even Chinese or Arabic alphabet if you want).

  • it needs to have seen a lot of data with patterns to generalize weights for transition from C->A->T, correct ?
    No, when you train on random text, the network is not using context of previous chars to predict current time step but it is still forced to predict each time step (slice of text) correctly because of the CTC loss. The CTC loss pushes the CNN to compute good features and the RNN to give good probabilities by character for all time steps.
    In other words, even with random text, the CTC loss still pushes the network to predict each char correctly independently of the other chars of the text.
    And actually a system trained on random text generalizes better on unseen words than a system trained on real words because the latter is biased toward words it has seen during training. For example a system trained on random text with Latin alphabet could read English, French and German but a system trained only on English words won't work well to read another language with same alphabet.

  • how can we ever generalize A623BBCEF ??
    Just because a network trained on random text learnt to read each char independently of other chars.

  • the output of such a RNN is a matrix containing character probabilities for each time-step" ..how do we calculate character probabilities ?
    The RNN takes the feature map for the current time step and compute a "score vector" from it. Sigmoid activation is used to map the scores to probabilities (between 0 and 1).
    In the end it is just a fully connected layer with as many units as characters with a sigmoid activation on each unit.

| improve this answer | |
$\endgroup$
  • $\begingroup$ ..appreciate it and am glad the model worked for you :) .. kindly clarify - when u mean random text, do u mean specifically alphanumeric strings ? ..and yes, i have read the above blog and the original paper and its quite clear, technically, that to find the "happy path" (pg5 /8 in the paper) the algo needs to generalize the transitions from label a to label b / blank ..this clearly means it needs to have seen a lot of data with patterns to generalize weights for transition from C->A->T, correct ? how can we ever generalize A623BBCEF ?? both with best path as well as prefix decoding $\endgroup$ – Vikram Murthy May 23 '19 at 3:36
  • $\begingroup$ sorry, just to add, i also read up on this repositum.tuwien.ac.at/obvutwoa/content/titleinfo/2774578 and it clearly mentions "If trained with the Connectionist Temporal Classification (CTC) loss function, the output of such a RNN is a matrix containing character probabilities for each time-step" ..how do we calculate character probabilities ? only by looking at lots of examples and then generalizing that if the previous 2 chars were "D" and "O" , "G" appears with 45% and "N" appears with 25% ..this can never happen for alphanumeric strings $\endgroup$ – Vikram Murthy May 23 '19 at 3:56
  • $\begingroup$ I updated my post to answer your questions. $\endgroup$ – Ismael EL ATIFI May 25 '19 at 23:08
  • $\begingroup$ appreciate the patient response :) $\endgroup$ – Vikram Murthy May 29 '19 at 7:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.