Questions tagged [overfitting]

Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.

0
votes
0answers
24 views

Is the model over-fitting the data?

On the y-axis you've got RMSE and on the x-axis you've got the number of epochs. Then in blue, the validation error, in red the training error. What do you think is the optimal number of epochs ...
1
vote
1answer
43 views

How to know if a model is overfitting or underfitting by looking at graph

Just recently got my hands on tensorboard, but can you tell me what features should I look for in the graph (Accuracy and Validation Accuracy) And please do enlighten me about the concept of ...
1
vote
1answer
43 views

Overfitting CNN models

I tried to develop a number of CNN architectures to train on a 1000-point subset of the "cat-dog" Kaggle training set (meaning, by the way, that all 1000 data points were labeled). I used a 700-150-...
-1
votes
0answers
15 views

“Debugging” overfitting results

I am trying to reproduce the results of a certain paper (on purpose, I am not stating which paper, as it is less relevant). I implemented the neural network's architecture, and was able to reproduce ...
1
vote
3answers
88 views

What is the point of getting rid of overfitting?

I'm having trouble understanding why I would use dropout, regularization, data augmentation, etc to get rid of overfitting in the first place. I get that if your model is too large or data is too ...
0
votes
0answers
10 views

Getting the right architecture with dynamic feature selection

I am building a NN (with keras) to address a problem that is mappable to the following: Each sample is composed of ~250 features of which ~100 should be used to determine the importance of the other ...
2
votes
1answer
138 views

Which model to chose based on learning curve

I trained my model using different regression techniques, and I'm not sure which model to choose based on the learning curve. 1) Should I choose Lasso, since train and CV converge at the end 2) ...
0
votes
0answers
11 views

How to deal with a situation that the number of features in training dataset is larger than the number of training examples

I am playing a Kaggle competition, Don't Overfit Ⅱ And I am dealing with a situation that the number of features in training dataset is larger than the number of training examples, which has 250 ...
0
votes
0answers
13 views

Which accuracies to report in this case?

I am new to ML research and to writing ML paper. An ML research project resulted in a family of algorithms $A_i$. These algorithms transform certain type of data. This data is fed into a neural ...
0
votes
1answer
31 views

Having averaged trials which are less than the number of features

Suppose I have an experiment where I have 70 features and 48 samples. The target variable is binary (0,1) and the 48 samples are divided such that 24 of them correspond to outcome 1 and the other 24 ...
0
votes
1answer
60 views

My model accuracy doesn't change after first epoch

I've created a model to predict housing prices in LA, and what should be a simple regression problem, is giving me headache because the loss is just too big and my accuracy wont change. I've already ...
2
votes
0answers
48 views

Knowing when a GAN is overfitting (sequence classification study)

I have sequences of long, sparse 1_D vectors (3000 digits, made of of 0s and 1s) that I am trying to classify. I have previously implemented a simple CNN to classify them with relative success (with ...
2
votes
1answer
74 views

How to recognise when to stop training based on Overfitting/Underfitting?

I am trying to train a LSTM network, over a total of 200 epochs, with hidden layer size of 100 and 1 dense layer after the LSTM layer. I have used a batch size of 10 for the same. Basically, I am ...
2
votes
0answers
86 views

Overfitting in K-means

How do you test your results for overfitting in a k-means run? Some people have said use a training set. I have about 1500 records and about 20 fields.
0
votes
1answer
231 views

Over fitting in Transfer Learning with small dataset

I am using Transfer Learning to perform image classification. Base model used : Resnet50 using ImageNet dataset ...
2
votes
1answer
201 views

Validation loss increases and validation accuracy decreases

I have an issue with my model. I'm trying to use the most basic Conv1D model to analyze review data and output a rating of 1-5 class, therefore the loss is categorical_crossentropy. Model structure is ...
7
votes
2answers
701 views

Validation vs. test vs. training accuracy. Which one should I compare for claiming overfit?

I have read on the several answers here and on the Internet that cross-validation helps to indicate that if the model will generalize well or not and about overfitting. But I am confused that which ...
0
votes
0answers
41 views

Remedies to CNN-LSTM overfitting on relatively small image dataset

Notes Using a pretrained model, trying data augmentation (not possible knowing nature of images, lowering number of parameters in the network, all didn't help) Context I have a sequence of images. ...
3
votes
1answer
124 views

Overfitting - how to detect it and reduce it?

I have a side project where I am doing credit scoring using R (sample size around 16k for train data and 4k for test data, and also another two 20k data batches for out-of-time validation) with ...
1
vote
2answers
180 views

SciKit-Learn Decision Tree Overfitting

I'm pursuing a computer science minor at my university, and one class I'm in is Machine Learning. We have a project to utilize a few algorithms we have learned so far. I've been using SciKit-Learn to ...
0
votes
0answers
11 views

How to apply curve fitting on both completed and active data?

Suppose we have a set of part failure times on which a specific curve (e.g. gamma distribution) is already fitted. Then another set of times are given where the part is still active (not failed yet). ...
1
vote
0answers
53 views

Network either overfits or underfits, but never generalizes - what to do?

I have a simple network with 1st level an LSTM, dropout, fully-connected and softmax layers; loss is cross-entropy (four classes, well balanced). Sequence length to LSTM is 172 samples, data is z-...
1
vote
2answers
48 views

Is there a disadvantage to letting a model train for a large number of epochs?

I created a model to solve a time series forecasting problem. I had a limited amount of time series with which I could train the model therefore I decided to augment the data. The data augmentation ...
1
vote
2answers
107 views

Is it bad to have a large gap between training loss and validation loss?

Say my training loss is 0.5 and my validation loss is 2.5 (both have stopped decreasing, validation loss never increased). I am clearly overfitting. If I add regularization, my training loss becomes 1 ...
1
vote
1answer
235 views

Retrain image classifier using MobileNet v2

I am using my own dataset to retrain mobilenet_v2_100_224 model, I currently have 4 classes where each class have more than 100 images still I'm observing overfitting even though I've used ...
3
votes
1answer
106 views

Bias-variance tradeoff in practice (CNN)

I first trained a CNN on my dataset and got a loss plot that looks somewhat like this: Orange is training loss, blue is dev loss. As you can see, the training loss is lower than the dev loss, so I ...
1
vote
2answers
94 views

How can I know if my NN TensorFlow model is overfitted or not?

I am new with TensorFlow (Python) and I can not juge my obtained results in terms of training and testing accuracy I am using the GradientDescentOptimizer with a learning coeff equal to 10^(-4) and ...
0
votes
1answer
29 views

Difference in labelling and normalizing train/test data

I am working on a dataset comprised of almost 17000 data points. Since it's a financial dataset and the components are many different companies, I need necessarily to split it by date. Therefore, ...
0
votes
1answer
27 views

Does running the script of train_test_split several times with varying test size in sklearn cause data leakage?

I ran a script of ridge and lasso regression twice with and without pca. Both times i got an okay R^2. but when i changed the train_test_split test size from 20 to 30%. My model started to over fit. ...
1
vote
2answers
78 views

How can someone avoid over fitting or data leak in ridge and lasso regression when the training score is high and test score is low?

I used the code provided here: https://towardsdatascience.com/ridge-and-lasso-regression-a-complete-guide-with-python-scikit-learn-e20e34bcbf0b The only difference is that i used StandardScalar on my ...
4
votes
3answers
211 views

Is Overfitting always bad?

I have a data set of total 8000 sound samples. These are the results of my multi layer neural network, binary classifier: ...
2
votes
0answers
57 views

How to improve a model with a high cross validation score yet with low accuracy on unseen data?

I have a model that is based on an experiment collected on 100 subjects. We are testing the model as follows: Record raw data from the subjects For each subject, compute the feature from the raw data ...
0
votes
1answer
19 views

What are some good design practices for creating/improving a CNN?

Recently I've been working on a mini side project in detecting age off of facial images. Aside from mistakes, I have made decent progress in creating my model. ...
1
vote
2answers
100 views

What are the possible approaches to fixing Overfitting on a CNN?

Currently I am trying to make a cnn that would allow for age detection on facial images. My dataset has the following shape where the images are grayscale. ...
1
vote
1answer
49 views

Not sure if over-fitting

I trained the data this way : There are four classes , the data distributed evenly (same amount of labels). Used min_max_scaler Used train_test_split(X,y,test_size=0.3,random_state=42,stratify=y) ...
0
votes
1answer
181 views

Check Overfitting in CNN

I am kind of new to NLP and text classification with Convolutional Neural Nets, and I have trained my first models quite recently. I am a little bit concerned with overfitting. I am doing multilabel ...
0
votes
1answer
18 views

Décision tree, How to see under/over fitting with just looking at the leafs?

My question is: how with just looking at the leafs of a decision tree could you tell if the model is under/over-fitting? Any sort of advice will be helpful.
3
votes
2answers
939 views

overfit a Random Forest

I am trying to overfit to the maximum a random forest classifier using scikit-learn to make some tests. Does somebody know what hyperparameters I can tune to do that? Or does somebody know which ...
0
votes
2answers
76 views

Is a Neural Network with 20 times the number of input neurons (on hidden layers) guaranteed to overfit? When is this not so?

I'm aware of the problem of over-fitting. One way to describe it is your Neural Network learning your training data to a high accuracy and performing poorly (generalizing) on new data. Was wondering ...
2
votes
1answer
1k views

Early stopping and final Loss or weights of models

In a deep model, I used the Early stopping technique as below in Keras: ...
2
votes
1answer
5k views

When should one use L1, L2 regularization instead of dropout layer, given that both serve same purpose of reducing overfitting?

In Keras, there are 2 methods to reduce over-fitting. L1,L2 regularization or dropout layer. https://keras.io/regularizers/ https://keras.io/layers/core/#dropout What are some situations to use L1,...
0
votes
1answer
66 views

How to do k-fold validation with classifiers?

I want to cross-validate a model that plays the card game below (see image). I trained the model on a dataset of 1000 games, with the goal to maximise the profit of each game. It works great on the ...
5
votes
1answer
40 views

How can I prevent this model to learn more(less) :)))

As you can see, GradientBoostingClassifier overfit with more training example. These are my parameter for the model: {'learning_rate': 0.1, 'loss': 'deviance', 'max_depth': 6, 'max_features': 0.3, '...
1
vote
1answer
266 views

What is Dropout in FC layer?

I know the purpose of dropout is to avoid overfitting by deactivating some neurons. However, I am interested in how it's done, ie: Math behind it, or intuition about why it works.
2
votes
0answers
42 views

Is my model overfitting when I add new features?

I'm working on simple 2-class classification problem. Nearly all features we have used (except one) are about the same for both classes: A random forest classifier confirms that one feature has an "...
2
votes
1answer
89 views

Convolution neural network with 11 million parameters unable to overfit on 100 image samples

I have been trying to do some sort of image enhancement on grayscale images. I have used both pixel wise loss and perceptual loss (perceptual loss uses classifier between 2 classes trained on the same ...
4
votes
2answers
83 views

overfit random walk using ANN in Keras

I am trying to build a neural net that will overfit random walk path. So, far I wasn't able to get a neural net that we shatter/overfit. I was wondering which parameters I should explore, or which ...
2
votes
2answers
416 views

Minimum number of samples to train XGBoost without overfitting

When using Neural Networks for image processing I learned a rule of thumb: to avoid overfitting, supply at least 10 training examples for every neuron. Is there a similar rule of thumb for XGBoost, ...
2
votes
1answer
58 views

Overfitted model produces similar AUC on test set, so which model do I go with?

I was trying to compare the effect of running GridSearchCV on a dataset which was oversampled prior and oversampled after the training folds are selected. The oversampling approach I used was random ...
3
votes
2answers
1k views

Too much inputs = overfitting?

First question : can I mix different sorts of inputs types for example, height and age (of course my inputs are normalized)? in general, can we mix different types of inputs in a neural network ? ...