0
$\begingroup$

I have two columns, a birth year and the first year of being a customer. There is a large number of samples that have a birth year less than the first year of being a customer which does not make sense in real life. For example, a customer is born in 1992 and was a customer since 1983.

I would like to know to can I solve this data preparation problem knowing that I cannot remove or swap the data in those two columns.

I appreciate any form of help.

$\endgroup$

1 Answer 1

1
$\begingroup$

Given that you cannot modify the data, I assume you are looking for data analysis techniques that can highlight a systematic pattern? If so we need more info.

Make use of context of the problem. Is the product young children or grown adults could buy? How was the data collected? Could some entries be from scammers or legacy software bugs? What fraction of inconsistent records do these represent? What other columns do you have about these customers? It really depends..

For example: If the data was manually entered or digitally scanned from physical documents it’s possible either year entry had “butter finger” or OCR translation errors. Perhaps you can semi infer other customers as family members based on their last name & other demographics. Perhaps clustering all customers reveals well defined sub groups between consistent & inconsistent records.

Be creative, formulate hypothesis & attempt to prove them with data to tell a plausible story. Make sure to analyze the consistent customers too in order to reject some hypothesis.

$\endgroup$
1
  • $\begingroup$ Thank you for your reply! So you re saying that I should solve this problem by making hypotesis and try to find some solutions to them ? I am very new to the data analysis field so in these cases I usually just delete any invalid data but this time it is almost 20% of the dataset ! P.S: the product is designed for adult customers so teens aren't the main target for the product. $\endgroup$
    – TheGame
    Dec 21, 2021 at 16:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.