1
$\begingroup$

I have been asked to unit-test my machine learning model(not the code that made the model). Since we wouldn't actually know what predictions models make, how to carry out the unit-testing to check the model's predictions against? How is this done?

EDIT 1:

The machine learning model I have is trained on tabular data of patients. let's take an example of cancer prediction(I am not allowed to disclose the actual one, but this example is very close). It takes multiple reading from various tests as inputs and outputs how close or how risky a patient is to get cancer.

EDIT 2:

Is there any way, like testing for range of value for every set of inputs (or) adversarial inputs(inputs that are sure model will fail on) or extreme input cases. What ate the best practices for this?

$\endgroup$
2
  • $\begingroup$ This is a very generic question, you need to specify what type of machine learning model it is and what type of input it takes. $\endgroup$ – RAVI TEJA M Oct 10 '20 at 7:01
  • $\begingroup$ @RAVITEJAM, do check my edit. $\endgroup$ – Naveen Kumar Oct 10 '20 at 7:05
1
$\begingroup$

In the above case, you can collect annotated data ( which is not seen by the model during training) and validate the predictions made by the model.

And another way is if you are a domain expert or have sufficient knowledge on the data, you can tweak the input values and test the predicted output with your expected output.

$\endgroup$
2
  • $\begingroup$ That's all. Isn't that something we all do to validate our model. Except for the input tweaking part. $\endgroup$ – Naveen Kumar Oct 10 '20 at 7:10
  • 1
    $\begingroup$ ML model can be testeded in different ways, 1. Interpreting the model (LIME/SHAP) 2. By changing your inputs by small amounts and observing the inputs. 3. Adding noise to the data and observing the inputs 4. Adversarial testing Based on your requirement you can pick any one of this. $\endgroup$ – RAVI TEJA M Oct 10 '20 at 7:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.