Most of the material I have read in the past usually assumes that the training set is flawless. However that doesn't seem to be the case here with what I am given. The data that is meant to send into the training set is often questionable (I don't even know how to start separating good from bad ones). I have spent much more time trying to pre-process the files in order to increase the validity of the data, than actually building the prototype.
I have only dumped a part of the data that is claimed to be better. There are still data from other sources to complement the missing part in the first set of training data. Given the amount of data, it is practically impossible for me to spend too much time on them.
So the question is, how do people deal with really horribly, inconsistent data in real life (assuming I am working with addresses, where people often write the wrong postcode due to bad city planning or lack of clear instruction, misspell the name of neighbourhood, plus data got "autocorrected" by excel such that "1-2" becomes "1-Feb", etc. etc.)? Specifically, how do I ensure the quality of the train classifier given the training set (and cross validation set) has questionable quality?