Questions tagged [overfitting]

Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

Threshold for overfitted models

It's common knowledge in DS that overfitted models perform well on training data and poorly on test data. But how do you decide if a model is really overfitting? I have nowhere (books, online courses, ...
0
votes
0answers
11 views

Does a CNN fully memorize ground truth if it has more parameters than training pixels?

ResNet consists of 25M trainable parameters. If only 30% of 600 $512 \times 512$ images is annotated, there are $600 * 512 * 512 * ~0.3 = 47,185,920$ ground truth pixels. A parameter is a floating ...
1
vote
0answers
27 views

How to avoid overfitting in Reinforcement Learning

I have implemented a RL model based on Deep Q-Learning for learning how to play a 2D game, like the ones in the OpenAI Gym. For testing the model, unlike most people, I have chosen to evaluate its ...
1
vote
1answer
21 views

Minimum Possible Test MSE

I have a little confusion. What follows is from Introduction to Statistical Learning (2013) by Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani. My understanding of what is going ...
0
votes
0answers
7 views
0
votes
0answers
9 views

Overfitting collaborative filtering

So I want to know whether or not my models are overfitting or the difference between train and validation errors are decent. $L$: is the number of neighbors The first column is the train error ...
0
votes
1answer
38 views

Can tuning epochs on validation set lead to overfitting on the training set?

I was recently working on a problem in which I split my data set into one training set (below called the "full training set") and a test set (with a 80%-20% split), and then split the full training ...
8
votes
3answers
138 views

Overfitting Question

Would you consider that overfitting?
1
vote
0answers
19 views

High Variance on CNN

I'm using a shallow CNN for my current project [this one]. I have a training dataset consisting of 1000 samples and a test dataset of 400 samples. I'm using the test dataset to choose the best ...
2
votes
1answer
32 views

Significant drop from validation accuracy to test accuracy

I am more familiar with classification tasks, though I have been working on a regression problem. I was given a large training dataset (>70k samples) and an independently collected test set (~2k). I ...
0
votes
0answers
19 views

CNN Architecture for Measuring Object Distances

I'm trying to use CNNs for infering object distances from an image. The input images correspond to states of a 2D game: Game states are not represented as images but as matrixes of observations. ...
1
vote
1answer
52 views

What is wrong with the below code? [closed]

I have been working on a project which i took from kaggle. I didn't get the result as mentioned in the website. What am I doing wrong here? ...
0
votes
0answers
19 views

At what value of Accuracy of train and test we can say Model is overfitting and Underfitting? [duplicate]

At what value of Accuracy of train and test we can say Model is overfitting and Under-fitting?
4
votes
1answer
77 views

Understanding why shuffling reduces weirdly the overfit

I am a student currently trying to create a classification model, however I am having difficulty understanding a weird overfitting problem. A dataset of about 30 000 entries, 30 features. The data is ...
4
votes
1answer
143 views

Interpretability of RMSE and R squared scores on cross validation

I'm working on a regression problem with 30k rows in my dataset, decided to use XGBoost mainly to avoid processing data for a quick primitive model. And i noticed upon doing cross-validation that ...
0
votes
0answers
14 views

How to gauge overfit with MLPClassifier and cross_val_score?

I'm learning sklearn. When using MLPClassifier.fit() and MLPClassifier.predict() I would ...
1
vote
1answer
37 views

Improving Accuracy of the Deep Learning Model

In my current project, I have only 647 rows (500 for training and 147 for testing) and I have applied the Keras Sequential model using the following code: ...
0
votes
0answers
38 views

My model is overfitting though i'v been using regularization techniques

I've been training an Xception model to recognize the disease of a plant from its leafs. So far i reached a training accuracy of 91% but the test accuracy is around 73%. So obviously my model is ...
0
votes
0answers
17 views

Autoencoder not overfitting data after large number of epochs and small number of samples

I am training a deep autoeocoder on numerical data, with python jupyter notebook. I have 17 samples, each with 534 values, and my auto encoder has all layers to 534, but even after 5,000 epochs, the ...
0
votes
0answers
24 views

Is the model over-fitting the data?

On the y-axis you've got RMSE and on the x-axis you've got the number of epochs. Then in blue, the validation error, in red the training error. What do you think is the optimal number of epochs ...
1
vote
1answer
95 views

How to know if a model is overfitting or underfitting by looking at graph

Just recently got my hands on tensorboard, but can you tell me what features should I look for in the graph (Accuracy and Validation Accuracy) And please do enlighten me about the concept of ...
1
vote
1answer
54 views

Overfitting CNN models

I tried to develop a number of CNN architectures to train on a 1000-point subset of the "cat-dog" Kaggle training set (meaning, by the way, that all 1000 data points were labeled). I used a 700-150-...
1
vote
3answers
100 views

What is the point of getting rid of overfitting?

I'm having trouble understanding why I would use dropout, regularization, data augmentation, etc to get rid of overfitting in the first place. I get that if your model is too large or data is too ...
0
votes
0answers
18 views

Getting the right architecture with dynamic feature selection

I am building a NN (with keras) to address a problem that is mappable to the following: Each sample is composed of ~250 features of which ~100 should be used to determine the importance of the other ...
2
votes
1answer
149 views

Which model to chose based on learning curve

I trained my model using different regression techniques, and I'm not sure which model to choose based on the learning curve. 1) Should I choose Lasso, since train and CV converge at the end 2) ...
0
votes
0answers
13 views

How to deal with a situation that the number of features in training dataset is larger than the number of training examples

I am playing a Kaggle competition, Don't Overfit Ⅱ And I am dealing with a situation that the number of features in training dataset is larger than the number of training examples, which has 250 ...
0
votes
0answers
13 views

Which accuracies to report in this case?

I am new to ML research and to writing ML paper. An ML research project resulted in a family of algorithms $A_i$. These algorithms transform certain type of data. This data is fed into a neural ...
0
votes
1answer
31 views

Having averaged trials which are less than the number of features

Suppose I have an experiment where I have 70 features and 48 samples. The target variable is binary (0,1) and the 48 samples are divided such that 24 of them correspond to outcome 1 and the other 24 ...
0
votes
1answer
201 views

My model accuracy doesn't change after first epoch

I've created a model to predict housing prices in LA, and what should be a simple regression problem, is giving me headache because the loss is just too big and my accuracy wont change. I've already ...
2
votes
0answers
99 views

Knowing when a GAN is overfitting (sequence classification study)

I have sequences of long, sparse 1_D vectors (3000 digits, made of of 0s and 1s) that I am trying to classify. I have previously implemented a simple CNN to classify them with relative success (with ...
2
votes
1answer
221 views

How to recognise when to stop training based on Overfitting/Underfitting?

I am trying to train a LSTM network, over a total of 200 epochs, with hidden layer size of 100 and 1 dense layer after the LSTM layer. I have used a batch size of 10 for the same. Basically, I am ...
2
votes
0answers
148 views

Overfitting in K-means

How do you test your results for overfitting in a k-means run? Some people have said use a training set. I have about 1500 records and about 20 fields.
0
votes
2answers
793 views

Over fitting in Transfer Learning with small dataset

I am using Transfer Learning to perform image classification. Base model used : Resnet50 using ImageNet dataset ...
3
votes
1answer
1k views

Validation loss increases and validation accuracy decreases

I have an issue with my model. I'm trying to use the most basic Conv1D model to analyze review data and output a rating of 1-5 class, therefore the loss is categorical_crossentropy. Model structure is ...
7
votes
2answers
1k views

Validation vs. test vs. training accuracy. Which one should I compare for claiming overfit?

I have read on the several answers here and on the Internet that cross-validation helps to indicate that if the model will generalize well or not and about overfitting. But I am confused that which ...
0
votes
0answers
64 views

Remedies to CNN-LSTM overfitting on relatively small image dataset

Notes Using a pretrained model, trying data augmentation (not possible knowing nature of images, lowering number of parameters in the network, all didn't help) Context I have a sequence of images. ...
3
votes
1answer
165 views

Overfitting - how to detect it and reduce it?

I have a side project where I am doing credit scoring using R (sample size around 16k for train data and 4k for test data, and also another two 20k data batches for out-of-time validation) with ...
1
vote
2answers
295 views

SciKit-Learn Decision Tree Overfitting

I'm pursuing a computer science minor at my university, and one class I'm in is Machine Learning. We have a project to utilize a few algorithms we have learned so far. I've been using SciKit-Learn to ...
0
votes
0answers
14 views

How to apply curve fitting on both completed and active data?

Suppose we have a set of part failure times on which a specific curve (e.g. gamma distribution) is already fitted. Then another set of times are given where the part is still active (not failed yet). ...
1
vote
0answers
57 views

Network either overfits or underfits, but never generalizes - what to do?

I have a simple network with 1st level an LSTM, dropout, fully-connected and softmax layers; loss is cross-entropy (four classes, well balanced). Sequence length to LSTM is 172 samples, data is z-...
1
vote
2answers
75 views

Is there a disadvantage to letting a model train for a large number of epochs?

I created a model to solve a time series forecasting problem. I had a limited amount of time series with which I could train the model therefore I decided to augment the data. The data augmentation ...
1
vote
2answers
233 views

Is it bad to have a large gap between training loss and validation loss?

Say my training loss is 0.5 and my validation loss is 2.5 (both have stopped decreasing, validation loss never increased). I am clearly overfitting. If I add regularization, my training loss becomes 1 ...
1
vote
1answer
443 views

Retrain image classifier using MobileNet v2

I am using my own dataset to retrain mobilenet_v2_100_224 model, I currently have 4 classes where each class have more than 100 images still I'm observing overfitting even though I've used ...
3
votes
1answer
159 views

Bias-variance tradeoff in practice (CNN)

I first trained a CNN on my dataset and got a loss plot that looks somewhat like this: Orange is training loss, blue is dev loss. As you can see, the training loss is lower than the dev loss, so I ...
1
vote
2answers
129 views

How can I know if my NN TensorFlow model is overfitted or not?

I am new with TensorFlow (Python) and I can not juge my obtained results in terms of training and testing accuracy I am using the GradientDescentOptimizer with a learning coeff equal to 10^(-4) and ...
0
votes
1answer
42 views

Difference in labelling and normalizing train/test data

I am working on a dataset comprised of almost 17000 data points. Since it's a financial dataset and the components are many different companies, I need necessarily to split it by date. Therefore, ...
0
votes
1answer
32 views

Does running the script of train_test_split several times with varying test size in sklearn cause data leakage?

I ran a script of ridge and lasso regression twice with and without pca. Both times i got an okay R^2. but when i changed the train_test_split test size from 20 to 30%. My model started to over fit. ...
1
vote
2answers
104 views

How can someone avoid over fitting or data leak in ridge and lasso regression when the training score is high and test score is low?

I used the code provided here: https://towardsdatascience.com/ridge-and-lasso-regression-a-complete-guide-with-python-scikit-learn-e20e34bcbf0b The only difference is that i used StandardScalar on my ...
4
votes
3answers
295 views

Is Overfitting always bad?

I have a data set of total 8000 sound samples. These are the results of my multi layer neural network, binary classifier: ...
2
votes
0answers
69 views

How to improve a model with a high cross validation score yet with low accuracy on unseen data?

I have a model that is based on an experiment collected on 100 subjects. We are testing the model as follows: Record raw data from the subjects For each subject, compute the feature from the raw data ...