48
$\begingroup$

In the context of Machine Learning, I have seen the term Ground Truth used a lot. I have searched a lot and found the following definition in Wikipedia:

In machine learning, the term "ground truth" refers to the accuracy of the training set's classification for supervised learning techniques. This is used in statistical models to prove or disprove research hypotheses. The term "ground truthing" refers to the process of gathering the proper objective (provable) data for this test. Compare with gold standard.

Bayesian spam filtering is a common example of supervised learning. In this system, the algorithm is manually taught the differences between spam and non-spam. This depends on the ground truth of the messages used to train the algorithm – inaccuracies in the ground truth will correlate to inaccuracies in the resulting spam/non-spam verdicts.

The point is that I really can not get what it means. Is that the label used for each data object or the target function which gives a label to each data object, or maybe something else?

Improve this question
$\endgroup$

3 Answers 3

Reset to default
43
$\begingroup$

The ground truth is what you measured for your target variable for the training and testing examples.

Nearly all the time you can safely treat this the same as the label.

In some cases it is not precisely the same as the label. For instance if you augment your data set, there is a subtle difference between the ground truth (your actual measurements) and how the augmented examples relate to the labels you have assigned. However, this distinction is not usually a problem.

Ground truth can be wrong. It is a measurement, and there can be errors in it. In some ML scenarios it can also be a subjective measurement where it is difficult define an underlying objective truth - e.g. expert opinion or analysis, which you are hoping to automate. Any ML model you train will be limited by the quality of the ground truth used to train and test it, and that is part of the explanation on the Wikipedia quote. It is also why published articles about ML should include full descriptions of how the data was collected.

Improve this answer
$\endgroup$
8
  • $\begingroup$ During training, can gt (e.g. in instance segmentation problems) be modified or created given the information obtained (e.g. from score maps) from the features? $\endgroup$
    – Alex
    Commented Feb 3, 2018 at 18:39
  • $\begingroup$ @Alex: Not usually. There might be some circumstances where one revised output or semi-automated process produces the ground truth for the next algorithm in a pipeline. However, if you are referring to an algorithm revising its own targets via some rule, then that's not usually considered a new ground truth - instead the ground truth would be the original segmentations provided for training. Any clever automated refinement would instead be part of the model. $\endgroup$
    – Neil Slater
    Commented Feb 3, 2018 at 19:40
  • $\begingroup$ A refinement with human interaction, or that referred original non-image data (e.g. some source images are generated using 3D model, so can create much better "true" segmentation) could be new ground truth. Although you would maybe want to separate the idea of generation 1 ground truth used to build the first model from generation 2 ground truth that has been through an iteration, and used to build a second model, even if the second model is the same architecture just trained on feedback. $\endgroup$
    – Neil Slater
    Commented Feb 3, 2018 at 19:42
  • $\begingroup$ 'trained on feedback' - close, but not exactly. If you've seen FCN model, the last layer is score map which is plugged in log softmax loss function together with the gt map. What I do is take the score map, extract some data from it (e.g. number of argmax binary blobs) and (somehow) modify the gt mask before plugging it in the loss function. How legit is this? $\endgroup$
    – Alex
    Commented Feb 3, 2018 at 21:02
  • $\begingroup$ @Alex: That's part of your model, and not new ground truth. Unless you decide, arbitrarily, that the goal of a new model is to learn your combined function. In which case it is the ground truth for the new model - however, you should most definitely note the complex source of this data, as it has been modified from the original measurement in an automated way. $\endgroup$
    – Neil Slater
    Commented Feb 4, 2018 at 9:00
6
$\begingroup$

This is a simplified explanation : Ground truth is a term used in statistics and machine learning that means checking the results of machine learning for accuracy against the real world. The term is borrowed from meteorology, where "ground truth" refers to information obtained on site.

How you get that ground truth : Many options but usually humans will evaluate each scenario and give the right labels to it. For example you can upload a group of images to trainingset.ai , label them (where is a car, where is a pedestrian and so on) and that will be your ground truth to evaluate or train your ai algorithm.

Improve this answer
$\endgroup$
0
4
$\begingroup$

Ground truth: That is the reality you want your model to predict.

It may have some noise but you want your model to learn the underlying pattern in data that’s causing this ground truth. Practically, your model will never be able to predict the ground truth as ground truth will also have some noise and no model gives hundred percent accuracy but you want your model to be as close as possible.

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.