# Data Loader (imbalanced classes) for Multitask Learning - same input multiple output

I am working on developing an MTL model. There are three tasks in the model, task1, task2, and task3. The classes in these tasks are imalanced and I am stuck at how I should sample from these so that the model doesnt overfit/underfit.

Task1 has 10 classes that are imbalanced.

Task2 has 5 classes, also imbalanced

Input:

The input are all images, and a single image has one class from task1 and one from task2.

Currently, I am using a custom made data loader which just shuffles the dataset, and is resulting in an unknown fit, where validation loss is lesser than the training loss and validation accuracy is higher than the training accuracy. There are no dropouts being used in the model. Here is the code for dataloader:

def get_data_generator(data, split ,batch_size=16):
data = data.sample(frac=1).reset_index(drop=True)
imagePath = ''
df =''

if split == 'train':
imagePath = '../MTLData/train/'
df = data[data.dir == 'train']
elif split == 'test':
imagePath = '../MTLData/test/'
df = data[data.dir == 'test']
elif split == 'vald':
imagePath = '../MTLData/vald/'
df = data[data.dir == 'vald']

pfrID = len(data.PFRType.unique())
ftID = len(data.FuelType.unique())
noxID = len(data.NOx.unique())
images, pfrs,fts, noxs = [], [], [], []
while True:
for i in range(0,df.shape[0]):
r = df.iloc[i]
file, pfr, ft, nox = r['Image'], r['PFRType'], r['FuelType'], r['NOx']

im = Image.open(imagePath+file)
im = im.resize((224, 224))
im = np.array(im) / 255.0
images.append(im)

pfrs.append(to_categorical(pfr, pfrID))
fts.append(to_categorical(ft, ftID))
noxs.append(to_categorical(nox, noxID))
if len(images) >= batch_size:
yield np.array(images), [np.array(pfrs), np.array(fts),np.array(noxs)]
images, pfrs, fts, noxs = [], [], [], []


Architecture:

I am using VGG16 for this classification and there are no dropout layers in the model.

Current performace:

Currently, validation loss is lesser than the training loss and validation accuracy is higher than training accuracy and the results do not make sense as towars the end of the training the validation loss and training loss change places, means validation loss starts increase. Here is the training graph: