Suppose, I have a dataset with the following features:
(FirstName, LastName, FullName, SchoolJoiningDate, SchoolGraduationDate)
My aim is to validate the data before feeding it into the ML pipeline. Some type of relationships which can exist on the columns of this dataset are:
- FullName = FirstName + LastName
- SchoolGraduationDate >= SchoolJoiningDate + 10 years
Similarly, there can other columns with different data types. I understand, the correlation between two numerical columns can give a sense of how the two numerical columns are related but the limitation of using correlation is as follows:
- Cannot help with the actual relationship between two columns (like addition, subtraction)
- Cannot be used for other non-numerical datatypes (like text, dates)
Is there any way I can detect such relationships between two columns?
