for an object detection task I created a small dataset to train an object detector. The class frequency is more or less balanced, however I defined some additional attributes with environmental conditions for each image, which results in a rather uneven distribution of classes depending on the viewed attribute (e.g. class X occurring with attributes A and B).
I used a typical training/validation split and the loss curve lets me conclude that no over-fitting is occurring. I am aware that in general for testing unseen images with a realistic feature distribution should be used, however due to the small size of the dataset, splitting off additional 10 or 20 % for testing or randomly drawing examples would result in certain combinations of classes and attributes not or only barely occurring in the test dataset and manually selecting examples would be very time-consuming.
I am therefore now wondering, if it reasonable to use the whole dataset (train+val) for testing and calculation of performance metrics as a compromise since I see no signs over-fitting?