I suggest that you look up one-class classification.
When we say classification, we by default talk about so-called discriminative classifiers. I.e., models where we assume each sample to belong to exactly one of the pre-defined classes.
This doesn't work well with your application:
- as you already say, what if an image is not of skin at all?
This "no-skin" class is ill-defined: besides not being skin, it could be anything. Such ill-defined classes are extremely difficult to train with a "normal" (discriminative) classifier because is is so hard to positively describe this class.
- Psoriasis does not prevent melanoma (actually, a quick internet search suggests that psoriasis patients have a higher cancer risk, including certain skin tumors): i.e. your disease classes are not mutually exclusive.
One-class classification relaxes this assumption: it models each class independently of all other classes, thus:
- it doesn't rely on having all classes that could appear in the samples pre-defined.
In your case: what about other skin diseases? They certainly exist - and unless you're looking at a differential diagnosis (I'm not a medical doctor - but I do have the impression that melanoma vs. psoriasis
- a sample may be predicted to belong to none of the known classes.
- a sample may be predicted to belong to more than one class (e.g. "normal skin" and "psoriasis" or even all three: the image may contain some normal and some diseased skin.)
- In general, one-class methods deal well with "not-classes", i.e. where we have a well-defined class and the rest is not well defined ("not psoriasis" - could be anything from imitation leather to some kind of dermatitis)
- Consider the difference between skin (as in: not artificial leather, not t-shirt, no kitten etc.) vs. normal skin (as in: no skin disease). You may want to set up 2 different classes for that.
A quick search got me to 2 papers on arXiv that may be a good starting point for deep learning one-class classifiers: Learning Deep Features for One-Class Classification and Anomaly Detection using One-Class Neural Networks
However, these advantages of one-class classifiers are not avalable for "free": comparing one-class and discriminative classifiers for the same situation, you usually need more training examples to get the same predictive performance for the one-class classifier.
Generally speaking, medical diagnosis distinguishes between "diagnosis" and "differential diagnosis". Differential diagnosis means that there is information already that excludes many things, and the remaining question is to decide between a known list of possible diseases.
While differential diagnosis can be handled with discriminative classifiers, other diagnoses usually call for one-class architecture.
One class methods rely exclusively on examples of the "in-class" for training. For verification/validation, however, you need to carefully chose the out-of-class test samples. The recommendation is to find samples of the out-class which are expected to be hardest to get right.
In your case, talking to the medical doctors about differential diagnoses of psoriasis and melanoma may reveal which other skin lesions look similar to your target classes.
In addition, as you say, it will be good to make sure your classifier works on a number of BS samples (non-human skin, leather, artificial leather).
Another consideration that is separate from the one-class vs. discriminative classification:
- Do you want to detect the actual lesions, or
- Do you want to classify a patient of having psoriasis (or melanoma) even if the image you have does not contain a lesion ("If you have skin like this, you probably have a lesion somewhere")
For both diseases this is thinkable: psoriasis is a systemic disease, so there could be specific characteristics of the skin even outside the lesion areas. Sun-exposure increases the melanoma risk - other parts of the skin may have characteristic changes of that photo-damage that are not yet cancer.