4
$\begingroup$

Assuming that one has a neural network capable of returning the numerical digit from a given image of size 28x28px.

How would one split an image of unknown size and an unknown amount of digits into a series of 28x28px images to feed to this network? (The order of the digits must be obtainable.)

For example:

String of digits

How can this be split up to:

enter image description here

Assuming that there are not always 5 digits and the initial image is not always the same size.

Firstly, my thoughts would be to create a secondary neural network. This neural network would output an (x, y) coordinate. This could be used to crop the image to 28x28px about this coordinate. However, this neural network would only be able to locate 1 digit at a time.

Secondly, another idea is that a series of random crops could be performed and then all given to the number recognising neural network. However, this would create a high error rate and the number recognising neural network has no way of telling that there was no valid number given (Unless that output was added). But more importantly the order of the digits would/could be lost.

I am struggling to find any resource explaining a possible solution. Google's house number recognition takes the entire image of digits and returns a value. This can be found in How Google Cracked House Number Identification in Street View and Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks.

$\endgroup$
2
$\begingroup$

Actually this is called text line extraction. What I'm going to tell you is inspired from the lectures of this scientist. For finding digits you don't need to design a network. You should extract them and then feed them to a network in turn.

First of all you have to read your image.

import cv2
import numpy as np
from matplotlib import pyplot as plt
%matplotlib inline
img = cv2.imread('./doc1.png')
plt.imshow(img)

enter image description here

Then you have to make your image as a binary array.

img.shape

(2360, 1649, 3)

img = cv2.imread('./doc1.png',0)
img.shape

(2360, 1649)

plt.imshow(img)

enter image description here

plt.imshow(img,cmap = 'gray')

enter image description here

The following code shows some of the lines of the document:

plt.imshow(img[900:1020,500:900],cmap = 'gray')

enter image description here

bimg=cv2.cvtColor(img[900:1020,500:900],cv2.COLOR_GRAY2RGB)
bimg.shape

(120, 400, 3)

Next, you have to find the lines in the image, then you have to find the characters.

enter image description here

enter image description here

enter image description here

enter image description here

enter image description here

enter image description here

enter image description here

enter image description here

enter image description here

enter image description here

enter image description here

enter image description here

enter image description here

enter image description here

Basically the code was completely clear to me. If you don't understand it, let me know.

$\endgroup$
3
$\begingroup$

In Image Processing, this task is known as localization. You basically want to localize each digit in the image and then use your digit recognizer over the digits. A cursory google search for digit localization in images gives me following papers which seem to be very helpful.

$\endgroup$
1
$\begingroup$

I have worked on a similar condition where I needed to separate each digit. I have done this using Image segmentation, segmenting only the white pixel column wise.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.