0
$\begingroup$

I have an image-segmentation model (.h5) which was trained using dice coefficient, recall and precision metrics.

The model was trained using the ImageDataGenerator class with the following code:

history = model.fit_generator(train_data,
                              steps_per_epoch=20,
                              epochs=200,
                              validation_steps=5,
                              validation_data=valid_data,
                              callbacks=callbacks_list)

Thus, I cannot derive, X_test and y_test in order to run functions such as model.evaluate(), model.predict() or classification_report from Sklearn which require X_test and y_test as arguments.

Is it possible to export a report with metrics for all the class and for each class separately using the trained model and without training the model again ?

UPDATE

Based on the answer of @Mohith7548, I'm trying to get x_test and Y_test data in order to run ImageDataGenerator().flow() function. To succeed that, I implemented the following code:

img_path =  "data/balanced_classes/validation/val_imgs"
label_path = "data/balanced_classes/validation/val_labels"
train_data = []
train_label = []
for i in os.listdir(img_path):
    img = io.imread(os.path.join(img_path,i))
    img = img / 255.
    img = trans.resize(img, (256, 256), mode='constant')
    train_data.append(img)

for l in os.listdir(label_path):
    label = io.imread(os.path.join(label_path,l))
    label = label / 255.
    label = trans.resize(label, (256, 256), mode='constant')
    train_label.append(label)

train_data = np.array(train_data)
train_label = np.expand_dims(train_label, axis=-1)

The shapes of X_test is (samples_num, 256, 256, 3) and of y_test is (samples_num, 256, 256, 1)

But when I'm running the following code:

metrics = model.evaluate_generator(test_data, steps=len(X_test) / batch_size)

I get the following error which is raised from the node of my loss function (TverskyLoss):

...
TP = K.sum((inputs * targets))
Node: 'TverskyLoss/mul'
2 root error(s) found.
  (0) INVALID_ARGUMENT:  required broadcastable shapes
     [[{{node TverskyLoss/mul}}]]
     [[assert_greater_equal_1/Assert/AssertGuard/pivot_f/_23/_55]]
  (1) INVALID_ARGUMENT:  required broadcastable shapes
     [[{{node TverskyLoss/mul}}]]
0 successful operations.
0 derived errors ignored. [Op:__inference_test_function_6827]

Here is the code from the TverskyLoss function that I use:

# Tversky loss function
ALPHA = 0.5
BETA = 0.5

def TverskyLoss(targets, inputs, alpha=ALPHA, beta=BETA, smooth=1e-6):
        
        #flatten label and prediction tensors
        inputs = K.flatten(inputs)
        targets = K.flatten(targets)
        
        #True Positives, False Positives & False Negatives
        TP = K.sum((inputs * targets))
        FP = K.sum(((1-targets) * inputs))
        FN = K.sum((targets * (1-inputs)))
       
        Tversky = (TP + smooth) / (TP + alpha*FP + beta*FN + smooth)  
        
        return 1 - Tversky

What am I doing wrong ?

$\endgroup$

1 Answer 1

1
$\begingroup$

Yes, it is possible to generate evaluation metrics for your image segmentation model without retraining it. One way to do this is to use the model.evaluate_generator() method, which allows you to evaluate the model using a data generator.

You can use the ImageDataGenerator class to generate a data generator for your test data, and pass it to the model.evaluate_generator() method along with the number of steps to use for evaluation. This will return a list of evaluation metrics, such as loss and any metrics that you specified during model compilation (e.g. dice coefficient, recall, precision).

Alternatively, you can also use the model.predict_generator() method to generate predictions for your test data using a data generator, and then use those predictions to calculate evaluation metrics using functions from scikit-learn or other libraries.

Here's an example of how you might use the model.evaluate_generator() method to generate evaluation metrics for your image segmentation model:

# Create a data generator for the test data
test_data = ImageDataGenerator().flow(X_test, y_test, batch_size=batch_size)

# Evaluate the model using the test data generator
metrics = model.evaluate_generator(test_data, steps=len(X_test) / batch_size)

# Print the evaluation metrics
print(metrics)

You can then use these evaluation metrics to generate a report with metrics for all classes and for each class separately, if desired.

$\endgroup$
1
  • $\begingroup$ thank you for your helpful answer. Could you be more specific in how to get the X_test and y_test ? $\endgroup$
    – Capdi
    Dec 21, 2022 at 8:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.