2
$\begingroup$

There are a million and one examples and tutorials on how to train up a neural network on the sample sets like the MNIST data and the CIFAR-10 data, but how does one go from the toy examples of recognising 200x200 clips each containing a single centred object to a real problem like finding CIFAR-10 category objects (the dog and the cat below) within a picture, like I presume Google does for their photo annotation.

enter image description here

Can someone describe how one might approach this leap from the classroom to the real world?

Improve this question
$\endgroup$
3
  • $\begingroup$ Possible duplicate of datascience.stackexchange.com/questions/11091/… $\endgroup$
    – Neil Slater
    Jul 15, 2016 at 6:37
  • $\begingroup$ One answer adds as a footnote: "There are potentially other approaches, such as using a more sophisticated network (maybe an RNN) to help guide the classifier on best places to check." - that's precisely what I'd like to find out about. Doing some sort of sliding window or ROI pre-processing is obvious, but I actually have a NN which does no pre-processing, but I don't understand how it does it. (In-house code, so I can't share here, unfortunately) $\endgroup$
    – Ken Y-N
    Jul 15, 2016 at 6:49
  • 1
    $\begingroup$ That was my answer, but I haven't studied the area well enough to know how it could work, or what images it would be viable in. In some cases, such as OCR on cheques, you can use non-ML image processing techniques to isolate boxes etc. In any case, if a specific approach interests you, you should re-word your question to narrow it down. That would stop it being a duplicate, too. $\endgroup$
    – Neil Slater
    Jul 15, 2016 at 6:55

1 Answer 1

Reset to default
1
$\begingroup$

This is a well defined problem called text spotting. There are numerous avenues to tackle this problem but most of the good ones are based on deep learning. Naively you would use a network like the one you trained on MNIST to slide over your input and see where it fires strongly to build up a string. This approach works reasonably well but to convolute this over your whole input image is extremely computationally expensive. A way that is actually used in practice is a two step process, first there is a network that is trained in localizing areas of interest, which are bounding boxes of parts where there might be texts and then use a more advanced network to grab the text. To my understanding this is also done in one network pass nowadays as opposed to querying on single patches. If you search for text spotting you will find a lot of nice papers and a thesis, mostly from the same guy.

http://www.mathstat.dal.ca/~hgu/Neural%20Comput%20&%20Applic.pdf

Improve this answer
$\endgroup$
5
  • $\begingroup$ Sorry, I've reworded the question to focus on images rather than digits, but that seems a useful link. $\endgroup$
    – Ken Y-N
    Jul 15, 2016 at 8:36
  • $\begingroup$ That is actually a very different problem, which I will answer in a bit $\endgroup$
    – Jan van der Vegt
    Jul 15, 2016 at 8:39
  • $\begingroup$ Potentially useful source material for an answer here: github.com/kjw0612/awesome-deep-vision#object-detection $\endgroup$
    – Neil Slater
    Jul 15, 2016 at 8:44
  • $\begingroup$ Let's say there were 2 cats and 1 dog, would you want to know there were 2 cats and 1 dog or that there is at least a cat and a dog in the picture? These are very different questions/networks again $\endgroup$
    – Jan van der Vegt
    Jul 15, 2016 at 8:54
  • $\begingroup$ . . . or where each cat and dog was in the image - by way of bounding boxes or object masks. Or converting image to semantics like cs.stanford.edu/people/karpathy/deepimagesent . . . I think the OP would be happy to see the bounding box version. $\endgroup$
    – Neil Slater
    Jul 15, 2016 at 10:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.