1
$\begingroup$

I have a dataframe like this:

import pandas as pd

df = pd.DataFrame({'incorrect': ['jak', 'mya', 'kfka'], 
                   'correct': ['jack', 'maya', 'kafka']})


incorrect  correct
jak        jack
mya        maya
kfka       kafka

I want to predict correct results from incorrect values with machine learning in python. Which model can I use for this case? Thanks.

$\endgroup$
  • $\begingroup$ Could you give some more detail about your dataset? $\endgroup$ – bkshi Feb 2 '19 at 19:54
  • $\begingroup$ The data contains user names and surnames. However, due to a problem, some letters are missing. I wrote the correct values manually. After that, I would like to fix the future problems with a model. I cannot share data because user information is confidential. $\endgroup$ – E. Zeytinci Feb 2 '19 at 20:21
  • $\begingroup$ Are the names from a fixed set or the can vary in the future? $\endgroup$ – bkshi Feb 2 '19 at 20:31
  • $\begingroup$ Names can vary in the future. $\endgroup$ – E. Zeytinci Feb 2 '19 at 20:34
1
$\begingroup$

The task that you are trying to do is really difficult to accomplish using machine learning as there is no pattern in how the corrected words are produced from incorrect words. One thing you can do is populate a dictionary of different names from the internet and then follow the suggestions provided here.

| improve this answer | |
$\endgroup$
1
$\begingroup$

It's unclear from your example how exactly you intend to predict if all you have are pairs of terms.
If you're assuming that you have all possible correct and incorrect pairs, then you can simply use your data as a hash table where each incorrect term is the key of a correct term.
Otherwise, assuming you have all the correct terms but you want to be able to handle slightly changed versions of the incorrect terms. You can define a similarity measure (function) and use that to find the closest incorrect term, in order to return the correct term for it.
If you don't have all the correct words, then it's a problem.
You should look for a vocabulary of 'legal' terms (or names in your case) and use that as your dictionary. There are some libraries in python that can do that for you, but as far as I know these are all based on regular dictionaries, like the ones used by LibreOffice for example (PyEnchant) is one example for that. If you're looking for a dictionary for names, you should perhaps try looking for a specific dataset that consists of names.

Otherwise your data should be in a different format, which will have more information in it, such as context for example.

One way to 'predict' correct words based on incorrect words is constructing a LM (Language Model) for each (correct and incorrect) and based on probabilities returning the ones with maximal likelihood, like a basic translation model. But this will only make sense if you have a lot of text (not unigrams) and each word appears several times, otherwise you don't really have a way to construct the LMs.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for your answer. I can not upvote because of low reputation. $\endgroup$ – E. Zeytinci Feb 2 '19 at 20:50
  • $\begingroup$ Don't worry about it, good luck with your task $\endgroup$ – Oleg Feb 2 '19 at 20:51

Not the answer you're looking for? Browse other questions tagged or ask your own question.