0
$\begingroup$

Baidu has answered this question empirically, but I don't have a good background in math so I don't understand the answer:

Many studies theoretically predict that generalization error "learning curves" take a power-law form, ε(m) ∝ αmβg. Here, ε is generalization error, m is the number of samples in the training set, α is a constant property of the problem, and βg = −0.5 or −1 is the scaling exponent that defines the steepness of the learning curve—how quickly a model family can learn from adding more training samples1. Unfortunately, in real applications, we find empirically that βg usually settles between −0.07 and −0.35, exponents that are unexplained by prior theoretical work.

Here's the same text as a screenshot in case any of the math notation doesn't display properly:

enter image description here

For example, for image classification on ImageNet:

The top-1 classification error exponent is βg = −0.309. On the other hand, the exponent for top-5 classification error is βg = −0.488.

How can this be expressed for a non-mathy layperson? For example, how much improvement in accuracy results from a 10x increase in training data?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.