unit-testing Machine Learning models

Question

I have been asked to unit-test my machine learning model(not the code that made the model). Since we wouldn't actually know what predictions models make, how to carry out the unit-testing to check the model's predictions against? How is this done?

EDIT 1:

The machine learning model I have is trained on tabular data of patients. let's take an example of cancer prediction(I am not allowed to disclose the actual one, but this example is very close). It takes multiple reading from various tests as inputs and outputs how close or how risky a patient is to get cancer.

EDIT 2:

Is there any way, like testing for range of value for every set of inputs (or) adversarial inputs(inputs that are sure model will fail on) or extreme input cases. What ate the best practices for this?

Answer 1

In the above case, you can collect annotated data ( which is not seen by the model during training) and validate the predictions made by the model.

And another way is if you are a domain expert or have sufficient knowledge on the data, you can tweak the input values and test the predicted output with your expected output.