Questions tagged [overfitting]

Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.

Filter by
Sorted by
Tagged with
1
vote
0answers
11 views

Will oversampling help with generalization (small imbalanced dataset)?

I have an imbalanced dataset (2:1 ratio) with about 60 patients and 80 features. I performed RFE + stratified cross validation to reduce the features to 15 and I get an AUC of 0.9 with Logistic ...
1
vote
0answers
24 views

How to interpret training results

I am working on an image similarity network. I have around 90,000 pairs of images contain an equal number of positive and negative samples. For learning the similarity between image pairs, I used the ...
30
votes
8answers
6k views

What would I prefer - an over-fitted model or a less accurate model?

Let's say we have two models trained. And let's say we are looking for good accuracy. The first has an accuracy of 100% on training set and 84% on test set. Clearly over-fitted. The second has an ...
2
votes
1answer
23 views

reduction of model accuracy while using PCA for a regression problem

I am trying to build a prection problem to predict the fare of flights. My data set has several catergorical variables like class,hour,day of week, day of month, month of year etc. I am using multiple ...
1
vote
2answers
130 views

What to choose: an overfit model with higher evaluation score or a non-overfit model with lower one?

For lack of a better term, overfit here means a higher discrepancy between train and validation score and non-overfit means a lower discrepancy. This "dilemma" just showed in neural network model I'...
2
votes
3answers
66 views

Is over-fitting a matter of features engineering or a matter of parameters set?

I am using sklearn package to make models. I tried randomly to set some paramater to a sklearn.ensemble.RandomForestClassifier ...
1
vote
3answers
28 views

Will the MAE of testing data always be higher than MAE of training data?

On the Kaggle Course Page the chart below shows that MAE of testing data is always higher than MAE of training data. Why is this the case? Is it only limited to DecisionTreeRegressor model? Or the ...
2
votes
1answer
78 views

Overfitting model

I'm training two ResNet models on an image dataset. The first one has been trained with random weights, while the other has been pre-trained on ImageNet before. The second model starts overfitting ...
1
vote
1answer
42 views

Training a Siamese Neural Network for object similarity assessment

I am training a Siamese neural network with pairs of similar and dissimilar objects. The features of the objects are binary data on whether they contain some properties or not (2048 features per ...
1
vote
0answers
15 views

Validation score during training and checkpoint is different in keras

I have a tabular data with about 1500 columns where every column except the 1st column is sparse. I am trying to train a Feedforward neural network (1 hidden layer with 32 neurons) for a binary ...
1
vote
1answer
42 views

Which combination of 3 hyperparameters to combat overfitting of a convolutional neural network?

I have a small dataset with which I want to train a CNN by using Data Augmentation. Since the CNN is overfitting due to the small data set, I would like to optimize some hyperparameters. However, ...
1
vote
2answers
33 views

k-fold cross validation with RNNs

is it a good idea to use k-fold cross-validation in the recurrent neural network (RNN) to alleviate overfitting? A potential solution could be ...
2
votes
1answer
29 views

Is not having overfitting more important than overall score (F1: 80-60-40% or 43-40-40)?

I've been trying to model a dataset using various classifiers. The response is highly imbalanced (binary) and I have both numerical and categorical variables, so I applied SMOTENC and Random ...
1
vote
1answer
23 views

Problem with overfitting for a CNN

I am doing image classification with a CNN and I am having trouble building a network that does not do overfitting. I have in my training set 2000 images of 4 classes, while in my test set I have 3038 ...
2
votes
2answers
178 views

How to build an overfitted network in order to increase performances

I am learning how to implement CNN, and searching on the internet I have found that a trick to design a good network is to first build it in such a way that it overfits, and then use regularization to ...
4
votes
1answer
42 views

Overfitting due to features correlating with training set generation rules

As background, I am using a Deep Neural Network built using Keras to classify inputs into 5 categories. The current structure of the network is: Input layer (~450 nodes) Dense layer (750 nodes) ...
1
vote
0answers
33 views

Conv Net Model is overfitting

So I made a convolution neural network to classify between different phonemes. My input datasets are a series of 0.4-second long spectrograms, the labels are each an individual phoneme that happens at ...
1
vote
1answer
19 views

On which step should use SMOTE technique for over sampling?

I want to use SMOTE technique for over sampling but I don't know on which step on pre-processing I should use it. My preprocessing steps are: Missing values Removing Outliers Smoothing Data Should ...
0
votes
0answers
31 views

One part of my loss function overfits. How do I fix this?

I am working on an object detection problem where the final loss that is being optimized is the sum of an L2 loss (for the error in the predicted w, h values), and three binary cross entropy losses (...
0
votes
1answer
60 views

Random Forest Overfitting, issues with mtry=1?

I am constructing what is known as an 'Expected Goals' model for football. This metric measures shot quality and a probability is assigned to a shot to achieve this, i.e. the chance a shot will be ...
3
votes
2answers
74 views

Am I overfitting my random forest model (more information in description)?

First off, sorry if this a novice question! Relatively new to all this stuff. Posted this in Stack Overflow and someone sent me here! Hope it's the right place. Anyway, I'm working with 22 datasets ...
0
votes
0answers
25 views

Neural Network is overfitting when using bigger dataset

I'm tring to train a model using CNN (supervised) to solve a binary classification problem. I have pretty big dataset containing 2 800 000 samples, each having 100+ features. Because training with ...
2
votes
2answers
43 views

If my model is overfitting the training dataset, does adding noise to training dataset help regularizing the machine learning model

I would like to know if this is a best practice or not. Can we add noise to the training data to help the model "fit less the training data"; as a result, hoping to generalize better on new unseen ...
0
votes
1answer
49 views

Decision tree and random forest over fitting

I am working on a real state data set to predict the price of buying a house in Dubai based on area, no.of bedrooms, number of baths and the town which the house is in. All variables are numerical ...
0
votes
1answer
49 views

Correctly evaluate model with oversampling and cross-validation

I'm dealing with a classic case of dataset with binary imbalanced target (event 3%, non event 97%). My idea is to apply some sort of sampling (over/under, SMOTE etc.) to address the issue. As I see, ...
2
votes
3answers
71 views

How to fix my high validation loss and inaccuracy

New to machine learning and tried to train my bird recognization model and found very high validation loss and inaccuracy. I'm using this dataset: http://www.vision.caltech.edu/visipedia/CUB-200-2011....
0
votes
0answers
13 views

Threshold for overfitted models

It's common knowledge in DS that overfitted models perform well on training data and poorly on test data. But how do you decide if a model is really overfitting? I have nowhere (books, online courses, ...
0
votes
0answers
11 views

Does a CNN fully memorize ground truth if it has more parameters than training pixels?

ResNet consists of 25M trainable parameters. If only 30% of 600 $512 \times 512$ images is annotated, there are $600 * 512 * 512 * ~0.3 = 47,185,920$ ground truth pixels. A parameter is a floating ...
1
vote
1answer
174 views

How to avoid overfitting in Reinforcement Learning

I have implemented a RL model based on Deep Q-Learning for learning how to play a 2D game, like the ones in the OpenAI Gym. For testing the model, unlike most people, I have chosen to evaluate its ...
1
vote
1answer
27 views

Minimum Possible Test MSE

I have a little confusion. What follows is from Introduction to Statistical Learning (2013) by Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani. My understanding of what is going ...
0
votes
0answers
11 views
0
votes
0answers
13 views

Overfitting collaborative filtering

So I want to know whether or not my models are overfitting or the difference between train and validation errors are decent. $L$: is the number of neighbors The first column is the train error ...
8
votes
3answers
149 views

Overfitting Question

Would you consider that overfitting?
1
vote
0answers
30 views

High Variance on CNN

I'm using a shallow CNN for my current project [this one]. I have a training dataset consisting of 1000 samples and a test dataset of 400 samples. I'm using the test dataset to choose the best ...
2
votes
1answer
65 views

Significant drop from validation accuracy to test accuracy

I am more familiar with classification tasks, though I have been working on a regression problem. I was given a large training dataset (>70k samples) and an independently collected test set (~2k). I ...
0
votes
0answers
32 views

CNN Architecture for Measuring Object Distances

I'm trying to use CNNs for infering object distances from an image. The input images correspond to states of a 2D game: Game states are not represented as images but as matrixes of observations. ...
1
vote
1answer
53 views

What is wrong with the below code? [closed]

I have been working on a project which i took from kaggle. I didn't get the result as mentioned in the website. What am I doing wrong here? ...
0
votes
0answers
22 views

At what value of Accuracy of train and test we can say Model is overfitting and Underfitting? [duplicate]

At what value of Accuracy of train and test we can say Model is overfitting and Under-fitting?
4
votes
1answer
149 views

Understanding why shuffling reduces weirdly the overfit

I am a student currently trying to create a classification model, however I am having difficulty understanding a weird overfitting problem. A dataset of about 30 000 entries, 30 features. The data is ...
4
votes
1answer
178 views

Interpretability of RMSE and R squared scores on cross validation

I'm working on a regression problem with 30k rows in my dataset, decided to use XGBoost mainly to avoid processing data for a quick primitive model. And i noticed upon doing cross-validation that ...
0
votes
0answers
44 views

How to gauge overfit with MLPClassifier and cross_val_score?

I'm learning sklearn. When using MLPClassifier.fit() and MLPClassifier.predict() I would ...
1
vote
1answer
45 views

Improving Accuracy of the Deep Learning Model

In my current project, I have only 647 rows (500 for training and 147 for testing) and I have applied the Keras Sequential model using the following code: ...
0
votes
0answers
55 views

My model is overfitting though i'v been using regularization techniques

I've been training an Xception model to recognize the disease of a plant from its leafs. So far i reached a training accuracy of 91% but the test accuracy is around 73%. So obviously my model is ...
0
votes
0answers
26 views

Autoencoder not overfitting data after large number of epochs and small number of samples

I am training a deep autoeocoder on numerical data, with python jupyter notebook. I have 17 samples, each with 534 values, and my auto encoder has all layers to 534, but even after 5,000 epochs, the ...
0
votes
0answers
24 views

Is the model over-fitting the data?

On the y-axis you've got RMSE and on the x-axis you've got the number of epochs. Then in blue, the validation error, in red the training error. What do you think is the optimal number of epochs ...
2
votes
1answer
537 views

How to know if a model is overfitting or underfitting by looking at graph

Just recently got my hands on tensorboard, but can you tell me what features should I look for in the graph (Accuracy and Validation Accuracy) And please do enlighten me about the concept of ...
1
vote
1answer
80 views

Overfitting CNN models

I tried to develop a number of CNN architectures to train on a 1000-point subset of the "cat-dog" Kaggle training set (meaning, by the way, that all 1000 data points were labeled). I used a 700-150-...
1
vote
3answers
114 views

What is the point of getting rid of overfitting?

I'm having trouble understanding why I would use dropout, regularization, data augmentation, etc to get rid of overfitting in the first place. I get that if your model is too large or data is too ...
0
votes
0answers
21 views

Getting the right architecture with dynamic feature selection

I am building a NN (with keras) to address a problem that is mappable to the following: Each sample is composed of ~250 features of which ~100 should be used to determine the importance of the other ...
2
votes
1answer
161 views

Which model to chose based on learning curve

I trained my model using different regression techniques, and I'm not sure which model to choose based on the learning curve. 1) Should I choose Lasso, since train and CV converge at the end 2) ...