20
$\begingroup$

In the context of Machine Learning, I have seen the term Ground Truth used a lot. I have searched a lot and found the following definition in Wikipedia:

In machine learning, the term "ground truth" refers to the accuracy of the training set's classification for supervised learning techniques. This is used in statistical models to prove or disprove research hypotheses. The term "ground truthing" refers to the process of gathering the proper objective (provable) data for this test. Compare with gold standard.

Bayesian spam filtering is a common example of supervised learning. In this system, the algorithm is manually taught the differences between spam and non-spam. This depends on the ground truth of the messages used to train the algorithm – inaccuracies in the ground truth will correlate to inaccuracies in the resulting spam/non-spam verdicts.

The point is that I really can not get what it means. Is that the label used for each data object or the target function which gives a label to each data object, or maybe something else?

$\endgroup$
17
$\begingroup$

The ground truth is what you measured for your target variable for the training and testing examples.

Nearly all the time you can safely treat this the same as the label.

In some cases it is not precisely the same as the label. For instance if you augment your data set, there is a subtle difference between the ground truth (your actual measurements) and how the augmented examples relate to the labels you have assigned. However, this distinction is not usually a problem.

Ground truth can be wrong. It is a measurement, and there can be errors in it. In some ML scenarios it can also be a subjective measurement where it is difficult define an underlying objective truth - e.g. expert opinion or analysis, which you are hoping to automate. Any ML model you train will be limited by the quality of the ground truth used to train and test it, and that is part of the explanation on the Wikipedia quote. It is also why published articles about ML should include full descriptions of how the data was collected.

$\endgroup$
  • $\begingroup$ During training, can gt (e.g. in instance segmentation problems) be modified or created given the information obtained (e.g. from score maps) from the features? $\endgroup$ – Alex Feb 3 '18 at 18:39
  • $\begingroup$ @Alex: Not usually. There might be some circumstances where one revised output or semi-automated process produces the ground truth for the next algorithm in a pipeline. However, if you are referring to an algorithm revising its own targets via some rule, then that's not usually considered a new ground truth - instead the ground truth would be the original segmentations provided for training. Any clever automated refinement would instead be part of the model. $\endgroup$ – Neil Slater Feb 3 '18 at 19:40
  • $\begingroup$ A refinement with human interaction, or that referred original non-image data (e.g. some source images are generated using 3D model, so can create much better "true" segmentation) could be new ground truth. Although you would maybe want to separate the idea of generation 1 ground truth used to build the first model from generation 2 ground truth that has been through an iteration, and used to build a second model, even if the second model is the same architecture just trained on feedback. $\endgroup$ – Neil Slater Feb 3 '18 at 19:42
  • $\begingroup$ 'trained on feedback' - close, but not exactly. If you've seen FCN model, the last layer is score map which is plugged in log softmax loss function together with the gt map. What I do is take the score map, extract some data from it (e.g. number of argmax binary blobs) and (somehow) modify the gt mask before plugging it in the loss function. How legit is this? $\endgroup$ – Alex Feb 3 '18 at 21:02
  • $\begingroup$ @Alex: That's part of your model, and not new ground truth. Unless you decide, arbitrarily, that the goal of a new model is to learn your combined function. In which case it is the ground truth for the new model - however, you should most definitely note the complex source of this data, as it has been modified from the original measurement in an automated way. $\endgroup$ – Neil Slater Feb 4 '18 at 9:00
1
$\begingroup$

Ground truth: That is the reality you want your model to predict.

It may have some noise but you want your model to learn the underlying pattern in data that’s causing this ground truth. Practically, your model will never be able to predict the ground truth as ground truth will also have some noise and no model gives hundred percent accuracy but you want your model to be as close as possible.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.