Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
Ex:
A1 A2 A3 Class attribute
---------------------------
X Y Z A
X Y Z B
P Q R C
Should we remove first and second training examples.
If yes, why? If we remove those we are loosing information that if inputs are (X,Y,Z) then class may be A or B but not C
If no, why not?
No, you should not remove them.
The problem in your case are not these individual samples. The real problem is that the attributes (variables) that you chose, i.e. A1-A2-A3, are not informative enough to accurately discriminate the samples. This is visible from the fact that the same input space (i.e. X-Y-Z) leads to different classes (A-B).
If you visualize this in a 3D plot with different colors for each class, you will see that the same datapoint will have two colors (!). This is because two datapoints fall exactly on top of each other and it is impossible to find a linear/nonlinear separation for them, no matter how hard you try.
Therefore, if you wish to increase the classification accuracy, you should pick other attributes/variables and not drop these individual samples; you have to change the way you look at the problem :)
Practically speaking, you can of course throw this data into a model and see what happens. If the case you describe is rare within the dataset, it might not pose a huge problem, maybe just some undefined random behaviour for that particular set of input. However if it happens a lot, the results will likely be less pleasing. You can try it with and without removing your duplicate cases (assuming you yourself know which ones to remove?) - then just compare results.
You could think about this issue as meaning that a function returns a value (your class attribute) from a distribution of possible answers. For example, that the inputs argument mean a value is selected from a discrete set of answers.
Methematically, your example doesn't lend itself to model fitting via a single function, e.g. using regression analysis, SVM models, neural networks and all other standard machine learning algorithms. The issue is that your data doesn't support the idea of a direct mapping, such as: $f : x \rightarrow y$. A function maps one argument X (or set of arguments) to another Y, but if you have a different output given identical input arguments, you are technically not talking about a function in the strictest sense. Since modelling is usually defining none other than a mapping from input to output, we are in essence defining a function.
Multi-value functions do exist - perhaps there are some clues about which methods to try. A quick scan of the Applications sections in that linked article show there are times when your problem exists. Perhaps there are some methods that can cover your specific use-case, but without more information about what kind of model and data you are working with, it is hard to give more concrete suggestions.
