Here's the description about the data usage for ILSVRC 2016
of ImageNet
. I've interpreted it with the table as follows,
Data Image Label Changed Release_Time
-----------------------------------------------------------------
training 1,200,000 1,000 no always public
validation 50,000 1,000 some upon registration
test 100,000 0 some final evaluation
My question is specific about the test data
. Is it chosen from the same set of 150,000 images since ILSVRC 2012
, although randomly ? If so, most of the test data
has been made public before the final evaluation phase of the challenge, how the result could remain fair enough ?