Manually creating plants images dataset for machine learning plant type classification and leaf segmentation

Question

A group I work with wants to create its own plants data set that will be used for multiple projects like plant type classification and leaf segmentation for starters.
They are willing to provide all the means necessary for the creation of this data set. A partner agriculture company is willing to assist with planting what ever is needed and have someone monitor and capture images every X duration as per requirements. Different growth stages are desired as well.

I was asked to write the requirements for how the images should be acquired, however I'm not quite sure of that myself. What are some key points one should know for such a task and are there any known guidelines or tutorials available for data sets creation?

Note that currently for this task only 1 plant will be captured in each image as the non-learning methods that will be used to label and segment the images support only 1 plant per image.

I was thinking that based on the time it takes a specific plant to fully grow (which is known, approximately), N images should be captured every X days covering 360° of the plant. Also, images are to be captured from different angles and distances.

Some questions I have in mind that I cant seem to answer yet,

• Number of images, the more the merrier as long as there are differences?
• What about the location of the (single) plant in the image? Is it crucial or could it be augmented after the data set has been acquired?
• What about camera resolution? is any decent camera sufficient for the task or should it be high end?

I'm still researching what other considerations there are and would appreciate any information in that matter

Answer 1

Yes, in deep learning (image classification) more is merrier and there are several ways to do this. The approach you have mentioned of taking N images every X days is ok if you have the workforce to complete this task. Another method would be to artificially blow up your dataset using image augmentation. This augmentation will including rotating the plants, cropping, blurring, zooming, and much more. There are several libraries available to do this task and you can increase the size of your dataset by many times.

The location of the plant inside the image is important. If the picture does not have anything important in it, then there is nothing that the model would gain from them. AUgmenting a useless image will create more useless images. The plant does not need to be in the center of the picture but the picture should contain something important i.e. a feature that can be helpful to classify that plant.

Any decent camera will be more than enough for the task as long as the classification does not need features like the patterns on leaf or edges or anything like that. any decent modern camera now a days is more than enough to capture these fine details.

Answer 2

Well, it all depends on what you plan to accomplish with this dataset:

• If you are planning on using drones to identify crops in the wild then you should use a drone to take the images and get images in the wild. (By in the wild I mean a free environment that differs from a lab or something like it)

• If you are planning on monitoring plants on a constrained environment using fixed cameras so you should gather images on this conditions.

As Rajat pointed, the camera you choose needs to be able to grab all the feature needed to the identification, maybe machine learning approaches can identify plants with way less features than human eye does but maybe it can not. Either way you should consult with someone with expertise on plants to decide what method you gonna use.

Numbers can't hurt:

Well, they can give you some extra work but the more the merrier. One thing you need to keep in mind is that datasets must be representative enough to enable learning methods to work on the real world scenario. For example, if plan on training a algorithm to classify images token from above there is no reason to train it with images from under the plants.

If you plan on using deep learning, (i.e. you have the time and resources to train it) you should gather a really large number of images or find a way to augment them with meaningful transformations. One way is to use simulated images, from a computer rendering (that is used in UnityEyes for example) and maybe augment it to be more realistic using GANs.

In short terms, my advice would be to formalize what you want this dataset for, consider what restrictions you can guarantee on the test environment and this should give you a list of what you need to create your training dataset.