Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I have performed logistic regression. And I am getting an accuracy of 77% with my current model. I divided my training set into cross validation set and train set. And I plotted a learning curve (graph of training examples vs cost function for train set and cost function of cross validation set). My learning curve is shown below -
What does it indicate? Since both the curves have a very less difference one is (0.51 and other in 0.52), Is my model biased, or is it correct?
Generally speaking, the "normal" shape of a learning curve (defined as a "plot of error vs training set size is known as a learning curve" (1)) is to observe an initially very low training error indicating that the model almost perfectly learns the small amount of training data while the test error will be high. When the amount of training data increases, the training error is expected to increase, too, as it becomes harder for the model to learn the increasingly complex data. At some point usually the training error stops increasing because data complexity, i.e. the number of distinct patterns in the data, does not increase further - even when adding more data.
In contrast, the test error is expected to be high in the beginning and then decrease when train and test data become more similar (since you're adding more training data). That is, in the beginning the model overfits (which is good news since it means that it's able to learn the data) and later train and test error ideally converge. This occurs when training and test become more similar.
This could look similar to this (The black horizontal line is the Bayes error) (1):
Or like this for a more complex model (1):
And that is similary to what your graph shows.
In contrast, a model which is not able to capture even simple patterns when the amount of training data is low could produce a learning curve like this (1):
A third scenario would be a very complex model showing a learning curve like this (1):
Compared to the first plot, the training error does not increase as fast since the more complex model is able to overfit data even when data complexity increases. But it also shows a very high test error in the beginning and a larger gap between train and test error for larger amounts of data.
This answer might also be interesting for you to read.
References: (1) Probabilistic Machine Learning: An Introduction p. 109-110
