1
$\begingroup$

I build model that on input have correct word. On output there is possible word written by human (it contain some errors). My training dataset looks that:

input - output  
hello - helo  
hello - heelo  
hello - hellou  
between - betwen  
between - beetween  
between - beetwen  
between - bettwen  
between - bitween

etc. During preprocessing I add a measure of the distortion of a word. Then I hardcoding letters for numbers. My current model's using CNN. The number of neurons of input is the same as the longest word in training dataset and the number of neurons of output is the same as the longest word in traning dataset. This model doesn't work as I excepted. Word on the output is not look as I except. eg.

input - output
house - gjrtdd

Question:

How can I build/improve model for this task? Is CNN a good idea? What other methods can I use for this task?

Improve this question
$\endgroup$
6
  • $\begingroup$ This task is called spelling correction or spell checking. There is a large body of work on it. $\endgroup$
    – Emre
    Commented Jun 4, 2018 at 0:52
  • $\begingroup$ Rather I say reverse spelling correction. My input is correct word, as an output I want not correct word. $\endgroup$
    – CezarySzulc
    Commented Jun 4, 2018 at 6:01
  • $\begingroup$ Have you checked my answer? You can train a Generator (in the GAN concept) to do exactly that. $\endgroup$
    – pcko1
    Commented Jun 4, 2018 at 16:25
  • $\begingroup$ Oh sorry. Once you have a spell corrected corpus, it is almost trivial to find the reverse map. If you collect statistics, you could also obtain a distribution over the candidates, since each input will correspond to multiple outputs. $\endgroup$
    – Emre
    Commented Jun 4, 2018 at 18:10
  • $\begingroup$ @Emre It's not so trivial. One letter can be changed in dependence of their neighborhood. So it is not one to one change. $\endgroup$
    – CezarySzulc
    Commented Jun 4, 2018 at 18:40

2 Answers 2

Reset to default
1
$\begingroup$

Try a totally different approach, using Generative Adversarial Networks.

For this purpose you need:

  1. A Generator
  2. A Discriminator

See the scheme (credit O'Reilly):

enter image description here

The "Real Images" block in the scheme should be your training dataset (or ground truth). The Generator should generate the distorted words and the Discriminator should verify if the word is "adequately" distorted, based on a criterion of your choice, which can be any similarity measure between known words (database) and the generated one. Both the Generator and the Discriminator get trained on-the-go while in the training phase and in the end you will have two trained networks, of which the Generator would be very useful for your purpose.

Helpful source: https://deeplearning4j.org/generative-adversarial-network

Improve this answer
$\endgroup$
5
  • $\begingroup$ Is GAN working with 1D? $\endgroup$
    – CezarySzulc
    Commented Jun 4, 2018 at 18:35
  • $\begingroup$ Yes, GAN is a concept. $\endgroup$
    – pcko1
    Commented Jun 4, 2018 at 20:16
  • $\begingroup$ GAN is unsupervised learning algorithm, so real images will be 'correct words'? Do I need my output (NOT correct words)? $\endgroup$
    – CezarySzulc
    Commented Jun 5, 2018 at 9:12
  • $\begingroup$ Exactly, I would use the real words as "real images" in the figure. The purpose of the Generator should be to produce imaginary words that comply with the characteristics that you want (similar up to a certain degree to real words). These characteristics (measure of similarity) will be learned by the Discriminator, which will help the Generator to improve itself. $\endgroup$
    – pcko1
    Commented Jun 5, 2018 at 9:16
  • $\begingroup$ GAN is semi-supervised, i.e. it needs samples of the "real" class and it will generate the "fake" class. $\endgroup$
    – pcko1
    Commented Jun 5, 2018 at 9:18
0
$\begingroup$

You can use a sequence-to-sequence model based on different RNN (LSTM for example) types. As far as I know, seq2seq models has been applied to spell checking. You should take a look to the existing work on spell checking and figure out how to use them in your problem.

Maybe you can treat correct word - misspelled word pairs as paraphrases and solve the problem as a paraphrase generation task.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.