I have the following problem, I have trained a CNN and I can evaluate the network in-sample. I want to use the trained model for the class prediction of images for which I have no ground truth. However, there are other features referenced to these images that I can implement in a regression model along with predicted labels to predict Y. The only way to evaluate somehow the CNN is to infer if the predicted labels have an effect on Y e.g. to evaluate the significance of the variable with the predicted classes or the performance of the entire regression model.
Since the regression is interpretable, I think the approach can be assigned to interpretable AI (@Nikos M. thank you). Unfortunately, I cannot assign the approach to any method/taxonomy of ineterpretable AI I have read so far. The regression model serves in my example as a kind of surrogate model. It is not a global surrogate because I do not use the predictions from the black box but use the trained CNN model to predict new unknown images in the "surrogate" model.
Can anyone tell me if this kind of evaluation method for machine learning models has already been described in the community and what is it called?