6
$\begingroup$

This is my problem description:

"According to the Survey on Household Income and Wealth, we need to find out the top 10% households with the most income and expenditures. However, we know that these collected data is not reliable due to many misstatements. Despite these misstatements, we have some features in the dataset which are certainly reliable. But these certain features are just a little part of information for each household wealth."

Unreliable data means that households tell lies to government. These households misstate their income and wealth in order to unfairly get more governmental services. Therefore, these fraudulent statements in original data will lead to incorrect results and patterns.

Now, I have below questions:

How should we deal with unreliable data in data science? Is there any way to figure out these misstatements and then report the top 10% rich people with better accuracy using Machine Learning algorithms? -How can we evaluate our errors in this study? Since we have unlabeled dataset, should I look for labeling techniques? Or, should I use unsupervised methods? Or, should I work with semi-supervised learning methods? Is there any idea or application in Machine Learning which tries to improve the quality of collected data? Please introduce me any ideas or references which can help me in this issue.

Thanks in advance.

Improve this question
$\endgroup$
2
  • $\begingroup$ you asked this on Stack Overflow and @MaximHaytovich gave you a good answer $\endgroup$
    – LinkBerest - SO sold our work
    Commented Jun 28, 2015 at 3:12
  • $\begingroup$ Add a noise term, and model it to account for the types of errors you observe. $\endgroup$
    – Emre
    Commented Jul 14, 2015 at 21:19

4 Answers 4

Reset to default
1
$\begingroup$

I have read @MaximHaytovich's answer and it is a good one. I would just like to suggest some further options, which were generalized in that answer as feature engineering. I would suggest trying to do the obvious first and analyze the data before transforming it to become ready for any machine learning algorithm. First look at the data and learn yourself and hypothesize about what patterns could be indicators of fraud. What is the nature of the data that you are 100% sure is reliable (it is mentioned in the question you posted that there is available data that is reliable) and how informative are they for the goal? The more familiar you are with the data, the better interpretation you can provide to any output you get from the methods you use. Try doing some outlier analysis on the incomes or any features that may be fraudulent that may make sense to do outlier detection on. Are there any suspicious missing values? Try doing clustering (one of the unsupervised algorithms you were talking about) to see if there are any in-betweens or irregularities.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for your reply and suggestions. I will try to provide a good sample in near future. $\endgroup$
    – Ardeshir
    Commented Jul 11, 2015 at 19:36
1
$\begingroup$

I can't see any way to magically identify unreliable data in this context. I'm no expert but 2 ideas spring to mind:

  • Domain knowledge: an expert on household income may be able to describe some general rules for identifying suspect data based on experience, and you could try to convert these into rules

  • Benford's Law - I don't know if it would be suitable for this scenario but Benfords Law has applications in detecting accounting fraud, based on the expected frequency of numbers appearing in numeric values. Probably of no use especially if incomes are reported in rounded figures (e.g. nearest 1000) but might be worth a look for inspiration.

Improve this answer
$\endgroup$
1
$\begingroup$

I'm not sure it is possible to generalise as to how people may mis-state financial information they report to governments.

This probably varies by country and hence culture and if the information is used directly to affect the welfare benefits received by that individual, it may not be.

If there is data on household income from a more trusted source, tax receipts for instance, that might be used to normalise the self reported data.

Improve this answer
$\endgroup$
0
$\begingroup$

I suggest outlier detection models for your case. In particular Local Outlier Factor.

Local outlier factor tries to detect samples that behave like an outlier within a group. It can be useful for your case.

I don't know what are the survey questions but for example, let's say living in downtown with 3 children and going a cinema 4 times in a week form a group of people. Let's say average income of the group is 150k usd per year. You saw a family in survey results in that group and said 65k usd per year, you can easily say that there might be a misstatement since families like that have higher income.

Local outlier factor provides you such anomalies. I hope its clear. You can find more information in Wiki and sklearn.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.