1
$\begingroup$

I have a custom image class (mainly borrowed from examples) that takes in an image of size 3x244x244 for use in a VGG model and returns augmented versions (rotations of 90,180,270 and an Hflip). Previously, I used this class to get one image and return that and the dataloader and everything worked as expected. When I try to output multiple image arrays, I'm not sure what the proper setup should be for use in the batching (I end up with a list of four image arrays) as one batch(?), which needs to be broken out into individual arrays (but it's still in the dataloader loop).

    def __init__(self, patch_data,file_count=1,transform=None, target_transform=None):

    def __getitem__(self, index):
        image = self.patch_data
        image.astype(float)

        #Normalize the data to 0,1 from 2^16
        image = self.image_normalize(image) #image/65535.0 #image_normalize(image)
        output90, output180, output270, outputflip = create_augmented_data(image,flip = 1,rot90=1,rot180=1,rot270=1)
        out_data = [output90,output180, output270, outputflip]
        sample = {"image": out_data} #, "label": label}
            #sample = file_name
        return sample

to run the data into the model, I have:

        for i, data in enumerate(dataloader_all, 0):
            inputs = data['image']#.type(FloatTensor)
            outputs = model_vgg16(inputs)

The conv2d sizing complains because it's setup to expect a tensor of 3x244x244, but it is getting a list of multiples of those. Since I'm in that dataloader for loop, I'm unclear on what might be needed to get all 4 arrays in the list to be properly batched into the model. Since this is using the eval mode of the model, I "think" I could gear it to run for each item in the list separately, but it seems that I had something wrong way before altering it that way. Any ideas?

$\endgroup$
3
  • 1
    $\begingroup$ Welcome to DatascienceSE, you don't want to randomly augment your images? You want to make batches according to a single image, i.e. for each image you want predictions on 4 augmented images from your model? $\endgroup$ Jun 2 at 7:24
  • $\begingroup$ Yes, I take in one image and create 4 (or more) transformed images and I'd like to use that 4 as a batch to the VGG model. I was able to engineer it so that I run VGG once per transformed file found (so one prediction per file), but I feel like there's some dataloader magic that I'm not taking advantage of. $\endgroup$
    – einsteinxx
    Jun 3 at 3:38
  • $\begingroup$ Do let me know if you are satisfied with the answer? If not I will try my best possible way to edit it. Please consider accepting the answer if it answers your question. $\endgroup$ Jun 4 at 14:05
0
$\begingroup$

You are almost there. You just need to perform the batch inference correctly. So, while model inference you need to convert the list of images to a single tensor, as follows -

for i, data in enumerate(dataloader_all, 0):
    inputs = torch.stack(data['image'])
    outputs = model_vgg16(inputs)

Your code might be creating a tensor of list of images, while the model expects tensor of tensors (tensor of image tensors). You need to give a tensor of shape (num_images,3,244,244) as an input to your model. Where num_images is batch-size (4 or more in your case). You should go ahead and try to print and see the tensor shapes.

Also, just make sure your dataloader is converting your images to tensor. I think you are already doing this because you mentioned you already made single image inference.

$\endgroup$
2
  • $\begingroup$ Thanks! That's what I needed, to know it's tensor of tensors and the shape. $\endgroup$
    – einsteinxx
    Jun 7 at 17:34
  • $\begingroup$ Glad to know. Please consider accepting the answer if it answers your question. $\endgroup$ Jun 8 at 15:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.