I am working on a skin disease classification problem where I have successfully created a classifier ( TensorFlow + Keras ) which can classify images of two skin diseases.

The sample image needs to be classified in this manner :

  1. Whether the sample is an image of the skin.
  2. Does the skin have any two of the diseases ( Melanoma or Psoriasis )
  3. If a disease is found, to which class does it belong (CLASS1: Melanoma or CLASS2: Psoriasis

How can create Classifiers which could carry out the following tasks? Do I need the image localization or CNNs or something like YoLo? What steps should I implement?

I have created a classifier ( with an accuracy of 96% ) which can classify an image of the two diseases efficiently. But it can't detect the presence of the disease ( Step 2 in the above task list ).


The other answers are correct, I just want to expand since you seem to wonder where step 1. fits in.

I think you should add yet another class called Unknown. This class will be able to tell that it is not human skin, but preferably it should be even more precise. It also should be able to tell if a picture is a good input for disease detection. You only want to make classifications on data that is "similar enough" to data you trained on.

Examples of pictures to train as Unknown could be:

  • Random pictures of whatever
  • Picture containing human skin but is too far away
  • Picture containing human skin but with a bad resolution / blurry
  • Picture containing human skin but with bad lighting
  • etc

When you got the negative data for the Unknown class your can use it in one of two ways. Either you train a general model that does everything in one or you train two specialist models.

General model

Build a general model for all your problems by making every distinction into classes. You can do this by adding another class of Unknown to the list @thanatoz wrote:

Healthy:   0
Melanoma:  1
Psoriasis: 2
Unknown:   3

Specialist models

Another approach is to built two models. One for skin detection and then one for disease detection, separating step 1 and step 2 in your list.

Skin detector - The first model will not know anything about diseases, it will only tell if a picture contains human skin or not. So the classes would be:

Unknown: 0
Skin:    1

Disease detector - This model is only activated if the skin detector has verified that you have a sample of representative human skin. This would then work the same as @thanaztoz answer with the classes being:

Healthy:   0
Melanoma:  1
Psoriasis: 2
  • $\begingroup$ I was thinking of a similar idea. Thanks for the help. :-) $\endgroup$ – Shubham Panchal Apr 16 at 11:10
  • 1
    $\begingroup$ While it is in theory possible to describe the "unknown" class, that is very difficult (as you already suggest: all kinds of objects are needed - and many of those as this unknown class spans huge parts of the possible input space). One-class (or unary) classifiers go a step further and avoid this ill-defined "not-class". They model only the (well defined) "positive" class(es) and assign as unknown everything that does not sufficiently resemble any of the positive classes. $\endgroup$ – cbeleites Apr 16 at 11:56
  • 1
    $\begingroup$ If I understand you correctly, you are suggesting something along the lines with making anomaly detection trained on only the positive images and using that instead of what I call Skin Detector? $\endgroup$ – Simon Larsson Apr 16 at 12:18
  • $\begingroup$ @cbeleites, nvm I saw that you added an answer that answered my question as well. Seems like a better first approach since you avoid gathering ill-defined data. $\endgroup$ – Simon Larsson Apr 16 at 12:35
  • 1
    $\begingroup$ @SimonLarsson: you'll have to have also data from the ill-defined class for verification/validation. But that needs a careful design: thinking how to both get a decent overall coverage of that ill-defined group plus probably a more in-depth examination of cases where it is important and/or particularly difficult to get right, e.g. inflammations of the skin other than psoriasis etc. $\endgroup$ – cbeleites Apr 16 at 12:53

I suggest that you look up one-class classification.

When we say classification, we by default talk about so-called discriminative classifiers. I.e., models where we assume each sample to belong to exactly one of the pre-defined classes.

This doesn't work well with your application:

  • as you already say, what if an image is not of skin at all?
    This "no-skin" class is ill-defined: besides not being skin, it could be anything. Such ill-defined classes are extremely difficult to train with a "normal" (discriminative) classifier because is is so hard to positively describe this class.
  • Psoriasis does not prevent melanoma (actually, a quick internet search suggests that psoriasis patients have a higher cancer risk, including certain skin tumors): i.e. your disease classes are not mutually exclusive.

One-class classification relaxes this assumption: it models each class independently of all other classes, thus:

  • it doesn't rely on having all classes that could appear in the samples pre-defined.
    In your case: what about other skin diseases? They certainly exist - and unless you're looking at a differential diagnosis (I'm not a medical doctor - but I do have the impression that melanoma vs. psoriasis
  • a sample may be predicted to belong to none of the known classes.
  • a sample may be predicted to belong to more than one class (e.g. "normal skin" and "psoriasis" or even all three: the image may contain some normal and some diseased skin.)
  • In general, one-class methods deal well with "not-classes", i.e. where we have a well-defined class and the rest is not well defined ("not psoriasis" - could be anything from imitation leather to some kind of dermatitis)
  • Consider the difference between skin (as in: not artificial leather, not t-shirt, no kitten etc.) vs. normal skin (as in: no skin disease). You may want to set up 2 different classes for that.

A quick search got me to 2 papers on arXiv that may be a good starting point for deep learning one-class classifiers: Learning Deep Features for One-Class Classification and Anomaly Detection using One-Class Neural Networks

However, these advantages of one-class classifiers are not avalable for "free": comparing one-class and discriminative classifiers for the same situation, you usually need more training examples to get the same predictive performance for the one-class classifier.

Generally speaking, medical diagnosis distinguishes between "diagnosis" and "differential diagnosis". Differential diagnosis means that there is information already that excludes many things, and the remaining question is to decide between a known list of possible diseases.

While differential diagnosis can be handled with discriminative classifiers, other diagnoses usually call for one-class architecture.

One class methods rely exclusively on examples of the "in-class" for training. For verification/validation, however, you need to carefully chose the out-of-class test samples. The recommendation is to find samples of the out-class which are expected to be hardest to get right.
In your case, talking to the medical doctors about differential diagnoses of psoriasis and melanoma may reveal which other skin lesions look similar to your target classes.

In addition, as you say, it will be good to make sure your classifier works on a number of BS samples (non-human skin, leather, artificial leather).

Another consideration that is separate from the one-class vs. discriminative classification:

  • Do you want to detect the actual lesions, or
  • Do you want to classify a patient of having psoriasis (or melanoma) even if the image you have does not contain a lesion ("If you have skin like this, you probably have a lesion somewhere")
    For both diseases this is thinkable: psoriasis is a systemic disease, so there could be specific characteristics of the skin even outside the lesion areas. Sun-exposure increases the melanoma risk - other parts of the skin may have characteristic changes of that photo-damage that are not yet cancer.
  • $\begingroup$ Thanks for the help @cbeleites. :-) $\endgroup$ – Shubham Panchal Apr 16 at 11:52
  • $\begingroup$ @ShubhamPanchal: welcome $\endgroup$ – cbeleites Apr 16 at 12:07

You have to have sample tissues from skins with Melanoma, Psoriasis and Healthy Skins.

As is understood in the question, you developed a solution which can distinguish with 96% accuracy between melanoma and psoriasis. You are missing from the possibility of distinguish between sick and healthy skins.

The model you are looking for is a multiclass classification. Where CLASS3: Healthy skin.

You are doing ok so far, you just need to include this category and images of healthy skins in your model.

  • 1
    $\begingroup$ That's right. But what is the user tries to click an image of something which is not human skin. How can I detect whether the image has skin ( healthy or diseased ) or not? $\endgroup$ – Shubham Panchal Apr 16 at 6:03
  • 1
    $\begingroup$ In real world diagnosis, normal skin isn't the only other important group of samples: what about people who have other skin diseases than psoriasis or melanoma? I'd guess that "this is not normal skin, but neither is it psoriasis nor melanoma" would be a practically important prediction. $\endgroup$ – cbeleites Apr 16 at 12:02

This is a simple problem of Multi-class classification having 2 classes:


You can build a custom classifier to detect these images or fine-tune a pre-trained model to detect the following diseases. In the last layer of the model, the model could be given a threshold value of confidence above which the prediction id to be kept or not.

Apart from these, as these disease are going to exist even if there is a small patch somewhere, what could be done is using TTA (Test Time Augmentation) to find the diseases. You can read more about TTA here but will need to figure out some way of implementing the same in Keras.

  • $\begingroup$ Thanks @thanatoz for the valuable help. Specially for test time augmentation of which I was unaware. $\endgroup$ – Shubham Panchal Apr 16 at 11:11
  • 1
    $\begingroup$ Sorry to -1, but this is not a simple multi-class problem. As OP says, they need to detect images that do not belong to any of the 3 well-defined classes, and melanoma and psoriasis are not mutually exclusive diseases. Plus, there is any number of skin diseases that are neither melanoma nor psoriasis. $\endgroup$ – cbeleites Apr 16 at 12:03
  • $\begingroup$ @cbeleites, I have taken care of the same (other cases) by mentioning the confidence threshold for the predictions. I do agree that there could be other medical complications that I could have been ignorant of. But I just followed a simple practice of not looking into the data before building the first classifier. Your suggestions are most welcomed. $\endgroup$ – thanatoz Apr 16 at 18:30
  • $\begingroup$ @thanatoz: my point is that you may be confident (or not), but that confidence can be totally off if fundamental assumptions built into the model are wrong (side note: read Kahneman on confidence). This is before ever looking into any data: this is about finding out the needs and characteristics of the application at hand. The data in itself will rarely reveal this. In my experience this type of information may be so obvious to cooperation partners that they never mention it. And not knowing enough about modeling, they won't stumble over it later. Result: bad model and noone realizes that. $\endgroup$ – cbeleites Apr 20 at 10:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.