Questions tagged [overfitting]

Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.

Filter by
Sorted by
Tagged with
35
votes
8answers
7k views

What would I prefer - an over-fitted model or a less accurate model?

Let's say we have two models trained. And let's say we are looking for good accuracy. The first has an accuracy of 100% on training set and 84% on test set. Clearly over-fitted. The second has an ...
10
votes
2answers
13k views

Validation vs. test vs. training accuracy. Which one should I compare for claiming overfit?

I have read on the several answers here and on the Internet that cross-validation helps to indicate that if the model will generalize well or not and about overfitting. But I am confused that which ...
9
votes
5answers
12k views

In which epoch should i stop the training to avoid overfitting

I'm working on an age estimation project trying to classify a given face in a predefined age range. For that purpose I'm training a deep NN using the keras library. The accuracy for the training and ...
7
votes
1answer
3k views

Why k-fold cross validation (CV) overfits? Or why discrepancy occurs between CV and test set?

Recently, I was working on a project and found my cross-validation error rate very low, but the testing set error rate very high. This might indicate that my model is overfitting. Why does my cross-...
0
votes
1answer
45 views

Learning curves

I am working on a multiclass classification problem. I want to know weather my model is overfitting or underfitting. I am learning how to plot learning curves and have 4 doubts. 1.) Is the ordering of ...
15
votes
2answers
11k views

Can overfitting occur even with validation loss still dropping?

I have a convolutional + LSTM model in Keras, similar to this (ref 1), that I am using for a Kaggle contest. Architecture is shown below. I have trained it on my labeled set of 11000 samples (two ...
6
votes
2answers
536 views

Why an increasing validation loss and validation accuracy signifies overfitting?

When I train a neural network, I observe an increasing validation loss, while at the same time, the validation accuracy is also increased. I have read explanations related to the phenomenon, and it ...
4
votes
1answer
458 views

Can a novelty detection model overfit?

Can a novelty detection model overfit? In novelty detection, the model is trained on normal data instances (not polluted by outliers) where no labels are used in the training process, while validated ...
6
votes
1answer
20k views

Validation loss increases and validation accuracy decreases

I have an issue with my model. I'm trying to use the most basic Conv1D model to analyze review data and output a rating of 1-5 class, therefore the loss is categorical_crossentropy. Model structure is ...
3
votes
2answers
6k views

Overfitting in an unsupervised technique

I am trying to understand if over-fitting can happen in an unsupervised technique like kmeans clustering. Could someone help me understand if and how this would happen? Thanks.
12
votes
4answers
11k views

How to know the model has started overfitting?

I hope the following excerpts will provide an insight into what my question is going to be. These are from here. The learning then gradually slows down. Finally, at around epoch 280 the ...
5
votes
2answers
5k views

Is Overfitting a problem in Unsupervised learning?

I come to this question as I read the use of PCA to reduce overfitting is a bad practice. That is because PCA does not consider labels/output classes and so Regularization is always preferred. That ...
5
votes
1answer
17k views

Accuracy and loss don't change in CNN. Is it over-fitting?

My task is to perform classify news articles as Interesting [1] or Uninteresting [0]. My training set has 4053 articles out of which 179 are Interesting. The validation set has 664 articles out of ...
4
votes
2answers
4k views

What are the possible approaches to fixing Overfitting on a CNN?

Currently I am trying to make a cnn that would allow for age detection on facial images. My dataset has the following shape where the images are grayscale. ...
6
votes
2answers
611 views

Can a linear regression model without polynomial features overfit?

I've read in some articles on the internet that linear regression can overfit. However is that possible when we are not using polynomial features? We are just plotting a line trough the data points ...
4
votes
1answer
932 views

Overfitting - how to detect it and reduce it?

I have a side project where I am doing credit scoring using R (sample size around 16k for train data and 4k for test data, and also another two 20k data batches for out-of-time validation) with ...
2
votes
2answers
241 views

What to choose: an overfit model with higher evaluation score or a non-overfit model with lower one?

For lack of a better term, overfit here means a higher discrepancy between train and validation score and non-overfit means a lower discrepancy. This "dilemma" just showed in neural network model I'...
8
votes
2answers
933 views

Is over fitting okay if test accuracy is high enough? [duplicate]

I am trying to build a binary classifier. I have tried deep neural networks with various different structures and parameters and I was not able to get anything better than ...
6
votes
1answer
605 views

Should I prevent augmented data to leak to the test/cross validation sets

I have been working with the cats vs dogs dataset from kaggle which consist on 25000 images of cats and dogs labelled accordingly (btw, great dataset, totally recommended!) One of the things I did ...
5
votes
2answers
579 views

Can we use a model that overfits?

I am on a binary classification problem with the AUC metrics. I did a random split 70%, 30% for training and test sets. My first attempts using random forest with default hyper-parameters gave me auc ...
2
votes
1answer
184 views

Feature selection where adding features are deteriorating model

k I am training a kNN classifier with 144 features and graphed the accuracy vs number of features used and got this. What might be the reason for the drops in the accuracy at some points of the graph?...
1
vote
1answer
73 views

Hyperparameter tunning for Random Forest- choose the best max depth

I'm trying to choose the best parameters for random forest model. For that goal I hae run my model in loop with only one parameter and each time I have changed the number for the parameter max depth. ...
1
vote
1answer
209 views

My LSTM can't reduce error down to zero, when overfitting

I have implemented LSTM in c++ which steadily decreases in error, but slows down at the certain error value. It also seems to predict most of the characters, but gets stuck and not able to correct ...
0
votes
1answer
45 views

Compare cross validation and test set results

I am having a hard time understanding the results of a cross validation test and a test run on a test set. First I made the following pipeline: ...