I have word images as below:
Let's say it's a
256x64 image. My aim is to extract the text from the image as
73791096754314441539 which is basically what an OCR does.
I am trying to build model which can recognise word from images.
When I am saying word it can be any of the follwing:
- Any dictionary word, non-dictionary word
- a-z,A-Z, special characters including
I have built a model (snippet because of company policies) in tensorflow as below:
inputs = tf.placeholder(tf.float32, [common.BATCH_SIZE, common.OUTPUT_SHAPE, common.OUTPUT_SHAPE, 1]) # Here we use sparse_placeholder that will generate a # SparseTensor required by ctc_loss op. targets = tf.sparse_placeholder(tf.int32) # 1d array of size [batch_size] seq_len = tf.placeholder(tf.int32, [common.BATCH_SIZE]) model = tf.layers.conv2d(inputs, 64, (3,3),strides=(1, 1), padding='same', name='c1') model = tf.layers.max_pooling2d(model, (3,3), strides=(2,2), padding='same', name='m1') model = tf.layers.conv2d(model, 128,(3,3), strides=(1, 1), padding='same', name='c2') model = tf.layers.max_pooling2d(model, (3,3),strides=(2,2), padding='same', name='m2') model = tf.transpose(model, [3,0,1,2]) shape = model.get_shape().as_list() model = tf.reshape(model, [shape,-1,shape*shape]) cell = tf.nn.rnn_cell.LSTMCell(common.num_hidden, state_is_tuple=True) cell = tf.nn.rnn_cell.DropoutWrapper(cell, input_keep_prob=0.5, output_keep_prob=0.5) stack = tf.nn.rnn_cell.MultiRNNCell([cell]*common.num_layers, state_is_tuple=True) outputs, _ = tf.nn.dynamic_rnn(cell, model, seq_len, dtype=tf.float32,time_major=True)
My current approach is to use take input a word image pass it through a
CNN extract high level image features, convert the image features to sequential data as below
[[a1,b1,c1],[a2,b2,c2],[a3,b3,c3]] -> [[a1,a2,a3],[b1,b2,b3],[c1,c2,c3]]
then pass it through a RNN(LSTM or BiLSTM), then use
CTC (Connectionist Temporal Loss) to find the loss and train network.
I am not getting results as expected, I wanted to know if:
- There is some other better way to do this task
- If I am converting features to sequence correctly
- Any research paper where something like this is done.