0
$\begingroup$

I am working on developing an MTL model. There are three tasks in the model, task1, task2, and task3. The classes in these tasks are imalanced and I am stuck at how I should sample from these so that the model doesnt overfit/underfit.

Task1:

Task1 has 10 classes that are imbalanced. enter image description here

Task2:

Task2 has 5 classes, also imbalanced enter image description here

Task3:

enter image description here

Task3 has seven classes, imbalanced.

Input:

The input are all images, and a single image has one class from task1 and one from task2.

Currently, I am using a custom made data loader which just shuffles the dataset, and is resulting in an unknown fit, where validation loss is lesser than the training loss and validation accuracy is higher than the training accuracy. There are no dropouts being used in the model. Here is the code for dataloader:

def get_data_generator(data, split ,batch_size=16):
data = data.sample(frac=1).reset_index(drop=True)
imagePath = ''
df =''

if split == 'train':
    imagePath = '../MTLData/train/'
    df = data[data.dir == 'train']
elif split == 'test':
    imagePath = '../MTLData/test/'
    df = data[data.dir == 'test']
elif split == 'vald':
    imagePath = '../MTLData/vald/'
    df = data[data.dir == 'vald']

pfrID = len(data.PFRType.unique())
ftID = len(data.FuelType.unique())
noxID = len(data.NOx.unique())
images, pfrs,fts, noxs = [], [], [], []
while True:
    for i in range(0,df.shape[0]):
        r = df.iloc[i]
        file, pfr, ft, nox = r['Image'], r['PFRType'], r['FuelType'], r['NOx']

        im = Image.open(imagePath+file)
        im = im.resize((224, 224))
        im = np.array(im) / 255.0
        images.append(im)

        pfrs.append(to_categorical(pfr, pfrID))
        fts.append(to_categorical(ft, ftID))
        noxs.append(to_categorical(nox, noxID))
        if len(images) >= batch_size:
            yield np.array(images), [np.array(pfrs), np.array(fts),np.array(noxs)]
            images, pfrs, fts, noxs = [], [], [], []

Architecture:

I am using VGG16 for this classification and there are no dropout layers in the model.

Current performace:

Currently, validation loss is lesser than the training loss and validation accuracy is higher than training accuracy and the results do not make sense as towars the end of the training the validation loss and training loss change places, means validation loss starts increase. Here is the training graph:

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.