How do you prepare the negative dataset for binary classification? Let us say that I am building a classifier that has to classify whether the input image is of a car or not. I already have a dataset that consists of thousands of cars. But what about negative classes? Should I collect any images which do not contain cars in them and label them as negative? How will I include negative classes in my dataset?
I think it is important to think about the application of that classifier and get the negative class images to be from a similar distribution as will be your application. For example if you want to classify blog images get the negative examples from blogs, if you want to classify facebook photos, get the facebook photos.
Note that this should apply also for your positive class (cars).
If you are not able to get a tons of photos from the distribution of your application you should definitely get them at least for your validation and test set.