I have word images as below:
Let's say it's a 256x64
image. My aim is to extract the text from the image as 73791096754314441539
which is basically what an OCR does.
I am trying to build model which can recognise word from images.
When I am saying word it can be any of the follwing:
- Any dictionary word, non-dictionary word
- a-z,A-Z, special characters including
spaces
I have built a model (snippet because of company policies) in tensorflow as below:
inputs = tf.placeholder(tf.float32, [common.BATCH_SIZE, common.OUTPUT_SHAPE[1], common.OUTPUT_SHAPE[0], 1])
# Here we use sparse_placeholder that will generate a
# SparseTensor required by ctc_loss op.
targets = tf.sparse_placeholder(tf.int32)
# 1d array of size [batch_size]
seq_len = tf.placeholder(tf.int32, [common.BATCH_SIZE])
model = tf.layers.conv2d(inputs, 64, (3,3),strides=(1, 1), padding='same', name='c1')
model = tf.layers.max_pooling2d(model, (3,3), strides=(2,2), padding='same', name='m1')
model = tf.layers.conv2d(model, 128,(3,3), strides=(1, 1), padding='same', name='c2')
model = tf.layers.max_pooling2d(model, (3,3),strides=(2,2), padding='same', name='m2')
model = tf.transpose(model, [3,0,1,2])
shape = model.get_shape().as_list()
model = tf.reshape(model, [shape[0],-1,shape[2]*shape[3]])
cell = tf.nn.rnn_cell.LSTMCell(common.num_hidden, state_is_tuple=True)
cell = tf.nn.rnn_cell.DropoutWrapper(cell, input_keep_prob=0.5, output_keep_prob=0.5)
stack = tf.nn.rnn_cell.MultiRNNCell([cell]*common.num_layers, state_is_tuple=True)
outputs, _ = tf.nn.dynamic_rnn(cell, model, seq_len, dtype=tf.float32,time_major=True)
My current approach is to use take input a word image pass it through a CNN
extract high level image features, convert the image features to sequential data as below
[[a1,b1,c1],[a2,b2,c2],[a3,b3,c3]] -> [[a1,a2,a3],[b1,b2,b3],[c1,c2,c3]]
then pass it through a RNN(LSTM or BiLSTM), then use CTC
(Connectionist Temporal Loss) to find the loss and train network.
I am not getting results as expected, I wanted to know if:
- There is some other better way to do this task
- If I am converting features to sequence correctly
- Any research paper where something like this is done.