0
$\begingroup$

I know that learning curves are a good tool to diagnose overfitting or underfitting for a model.

Their working principle is simple: The training/validation loss/accuracy is plotted depending on specific parameters, e.g., the learning rate, the model architecture complexity, the training set size, the number of training epochs, etc.

However, should I always plot many different learning curves, i.e., depending on these different parameters? Or is it sufficient, if I only plot one learning curve depending on just one parameter? Thus, would it be reliable enough, if I would just plot the learning curve based on the keras (tensorflow 2 as the backend) model.fit history, i.e., the training/validation loss depending on the training epochs.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Overfitting is a result of the entire model. It is best to visualize every part of the model to understand how the combination of elements is overfitting.

Ideally, you should visualize all elements on the same interactive figure so how the combination is overfitting. If that is not possible or the interpretation is difficult, then many single element visualizations would be an option. However, that increases the possibility of missing interactions causing overfitting.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks! So as the best solution you propose to make many many plots where all such parameters vary, right? I.e., if I have 3 parameters all with a range of [0, 10], I would get 10^3 plots. $\endgroup$
    – a.user
    May 10, 2020 at 14:12
  • $\begingroup$ Another possibility is a single 3D plot with the axes ranging from [0,10]. The number of dimensions for the plot depends on how overfitting is measured. $\endgroup$
    – Brian Spiering
    May 10, 2020 at 14:21
  • $\begingroup$ In my case - and AFAIK this is always the same, isn't it? - overfitting is measured by the gap between the training loss being very small and the validation loss being very large. What do you exactly mean with your statement: "The number of dimensions for the plot depends on how overfitting is measured." $\endgroup$
    – a.user
    May 10, 2020 at 14:26
  • $\begingroup$ Most plots should be 2-3 dimensions. It may or may not be useful to have overfitting take up a dimension. Overfitting can be view as a function, the rate of overfitting takes up a dimension. If you create a threshold for overfitting that transforms overfitting to a scalar, then overfitting as a dimension can be dropped from the visualization. $\endgroup$
    – Brian Spiering
    May 10, 2020 at 15:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.