My dataset is 70k images of people wearing clothes. Images are labeled: bbox position and class. There are 10 classes. I did 80:20 split. Categories are balanced with exception of one category, but I can accept poor performance on one category.
Goal is cloth recognition in images. When I feed image of person wearing pants and tshirt, I want to see two bboxes of these clothes.
I already trained few models from tf model zoo. I did over 100k steps on ssd mobilenet v1 and faster rcnn resnet 101.
Problem with ssd is that it won't converge. Loss is not getting below stable 2 and accuracy is bad. Problem with faster rcnn is that loss is below 1 but it's varying a lot and sometimes it jumps over 1.
What I've done:
I tried different batch sizes for ssd with no luck. FRCNN is locked with batch size 1. I improved dataset multiple times. I went from 50 unbalanced classes to 10 balanced classes. I didn't tweak hyperparameters from models configs besides batch size.
My access to strong gpu is limited for me so I can't just randomly try different hyperparams combinations with hope that it will work. Could you suggest me few things that I can do in order to improve my models? I would be very thankful.