I have digit images as below which I would like to identify:

enter image description here

Some are of slightly worse quality :

enter image description here

The images are not of a fixed resolution but are mostly in the range (80*20 to 130 *40). Due to lack of enough labelled data(~3K rows), I had to go for an open source digit dataset. I tried both MNIST and SVHN with not much luck. I've detailed both my approaches below..

Note : The only preprocessing that I've used is converting to grayscale ,resizing the images to a fixed size and doing a mean subtraction on the image.

1. SINGLE DIGIT MNIST and SLIDING WINDOW FOR TESTING : I trained a Convnet on the MNIST set with ~99% on the validation set.Then I used a sliding window to move over my test image and predict for each. This approach had two problems : First, as my images are not of a fixed resolution often I couldn't get the right sliding window (one which contains just one digit completely). Second, I cropped out individual digits from my test images, and tested my MNIST model on these, the results were 'wildly off', out of the ~15 samples that I had cropped out, my MNIST model couldn't get even one right. Here's a cropped out image for reference.

enter image description here

  1. SINGLE DIGIT SVHN I thought that the performance of my MNIST model on my test set maybe bad as these two are different type of images, numbers in the SVHN dataset are more similar to the ones I posted. I repeated the same single digit exercise with SVHN, with far better results, on my small cropped out set of 15 numbers,this time I got 10 right but the sliding window problem still remained.

  2. MULTI DIGIT SVHN : As house numbers in SVHN are of smaller lengths(max 5) than my numbers, I created 50k numbers of length 8 by horizontally stacking different numbers from the SVHN multi digits dataset.The best validation loss I've been able to achieve from this model is 6.70, not very great and obviously not very effective at predicting. I used a ConvNet with the following architecture:

model_input = Input(input_shape)
  x = Conv2D(32, (3, 3),data_format='channels_first', activation='relu', name='conv_32_1',padding='same')(model_input)
  x = Conv2D(32, (3, 3), data_format='channels_first',activation='relu',name='conv_32_2',padding='same')(x)
  x = MaxPooling2D(pool_size=(2, 2))(x)    
  x = Dropout(0.35)(x)

  x = Conv2D(64, (3, 3),  data_format='channels_first',activation='relu',name='conv_64_1',padding='same')(x)       
  x = Conv2D(64, (3, 3), data_format='channels_first',activation='relu',name='conv_64_2',padding='same')(x)    
  x = MaxPooling2D(pool_size=(2, 2))(x)    

  x = Conv2D(128, (3, 3), data_format='channels_first',activation='relu',name='conv_128_1',padding='same')(x)    
 x = Dropout(0.35)(x)

 x = Flatten()(x)

 x = Dense(768, activation='relu')(x)    
 x = Dropout(0.50)(x)
 output_list = [Dense(11, activation='softmax',name='digit_'+str(i))(x)    for i in range(1,9)]
 adam = Adam(lr=0.0005)

 model = Model(inputs=model_input, outputs=output_list)

 model.compile(loss='categorical_crossentropy', optimizer=adam, metrics=['accuracy']).

And this is the loss curve: enter image description here

My questions,going from the last approach to the 1st :

  1. Is my MULTI SVHN model too small for this problem/dataset? If not, how do I improve its performance?
  2. Is there a way to generate better candidate sliding windows, as my single digit SVHN seems to be the best model I have among the three.
  3. Is my reasoning correct that the MNIST model didn't work well on my images as the digits are somewhat different (handwritten vs computer generated) ?
  4. Are there other preprocessing techniques that I could use to improve my performance espeically for images like the 2nd one (18458882)?
  5. Is there any other approach that I could try to solve this?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.