I've come across a problem in assessing the design of a Machine Learning solution I am involved in, which I will describe using a commonly known dataset, the Iris Dataset, and some drawings.
Suppose you have training information about the 3 flower classes contained in the Iris Dataset, which means you can train a multivariate classifier that can distinguish between these 3 flower classes.
However, my dataset 4 classes of flowers, i.e. one extra class that the classifier hasn't been trained on. This dataset I call the "Main" dataset. The classification is guaranteed to pick one of the 3 Iris Dataset classes, meaning you 100% misclassify any instance's of the 4th class not included in training.
The objective I have is to figure out the Main dataset's composition in the 3 Iris flower classes. I am trying to do this with a second dataset, which I call the "Auxiliary" dataset, containing ONLY the 4th extra class, with a size equal to the expected number of the 4th class's instances in the 4-class dataset. Thus, I get a picture of the possible outcome of using the Iris Classifier on the 4th flower class.
Using the above, is it safe to infer the composition of the Main dataset in the 3 Iris classifier classes, by subtracting the composition of the instances based on the 4th class auxiliary dataset?
"Main" composition in Iris classes = "Main" output composition - "Auxiliary" output composition
Is there perhaps any literature relative to this?
