I am trying to develop a deep learning model that given an image of a car, it detects a car's make and model among 50 different brands, each with say another 50 models. What approach is probably the most suitable with highest accuracy for this problem?

I was thinking to use a YOLO detection method but not sure if this is best.


1 Answer 1


For detecting the make and model of cars from images with high accuracy across a large number of classes, I would recommend a convolutional neural network (CNN) architecture tailored for fine-grained image recognition.

Some key elements to consider:

  1. Use a pre-trained model like ResNet50 or VGG16 pre-trained on ImageNet as the base model. This initializes the model with learned feature maps helpful for image recognition tasks.

  2. Add custom classification layers on top to classify the 50 makes x 50 models = 2,500 classes. These layers should have enough capacity to differentiate the fine-grained details. Use a dataset of car images labelled with make/model for training. Consider augmenting it with rotations, crops, lighting changes etc to increase diversity.

  3. Fine-tune just the classifier portion first. Then fine-tune some of the latter convolutional blocks along with the classifier for better specialization.

  4. Use softmax loss or adaptations like label smoothing regularization that can help for many fine-grained classes.

Overall accuracy rates of over 90% should be possible for 2500 classes with a tuned CNN approach. YOLO is more suited for localization/bounding box detection - useful if you want to detect the boundary of cars too. But for pure make/model classification, a regular CNN would be more accurate.

There are some good open-source datasets for training a model to classify car make and model:

  1. Stanford Cars Dataset: Contains 16,185 images of 196 classes of cars. It has less diversity but high image quality and standardized cropping.

  2. CompCars Dataset: It contains data for both car classification and fine-grained recognition tasks. For classification, it has 431 car makes with 34,991 images. For fine-grained recognition, it has 13,858 images of 171 car models from the 431 makes.

  3. VehicleID Dataset: A large dataset with 221,763 images of 26,267 vehicles captured by surveillance cameras. It has less standardized images but huge diversity.

  4. Car Make and Model Recognition Dataset: Created from Google image search results, it contains 63,000 low-resolution images across 183 car makes and models. Helpful for evaluating real-world performance.

Some pre-processing is needed for alignment, cropping, colour correction etc. For 50 make x 50 models = 2500 classes, you may need to consolidate some granular classes from these datasets and augment more data for less represented classes.

Start with CompCars or VehicleID dataset as they are larger and closer to your needs. Fine-tune on the Stanford Cars dataset for better performance.

I hope it helps!

  • $\begingroup$ Thanks. I would need to detect the color of car as well. Any ideas there? I was thinking to use Yolo so to get bounding boxes and get the average color of the bbox. $\endgroup$
    – Mary
    Dec 1, 2023 at 4:43
  • 1
    $\begingroup$ I came up also with the idea that you can experiment not with 2500 (50*50) classes, but with 100 (50+50). It might be simpler for your model to learn it, as you would have more entities per class. You could also check whether brands correspond with models correctly which could be some additional quality metric. $\endgroup$ Dec 1, 2023 at 7:26
  • $\begingroup$ You can use the CompCars dataset which also contains some color labels for instance. You can use the same approach using a CNN. $\endgroup$
    – Multivac
    Dec 1, 2023 at 15:41
  • $\begingroup$ Yeah I just got the compCar dataset. Looks like it contains older cars till 2015. $\endgroup$
    – Mary
    Dec 1, 2023 at 17:51
  • $\begingroup$ Oh, it may be a problem if you are to use your model in more recent cars. Could you maybe create separate models one only for color detection using the compCar dataset and the other trained in a dataset with newer models for model and make detection? $\endgroup$
    – Multivac
    Dec 1, 2023 at 18:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.