Questions tagged [overfitting]

Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.

Filter by
Sorted by
Tagged with
0
votes
1answer
22 views

How many ways are there to check model overfitting?

I am running xgboost on a regression classification problem where the model is predicting a score of how likely a gene is to cause a disease from 0-1. I try to avoid overfitting in all the ways I can ...
0
votes
0answers
8 views

Why is my time series model predicting strange results?

I am trying to predict some time-series data. The output data predicts two numbers (one that's usually greater than 1 and another that is usually less than 1). I've plotted about 800 samples where the ...
0
votes
0answers
10 views

How can a CNN account for spectro-temporal constraints in neural data?

What are there the best ways to leverage the unique "geometrical" constraints of spectro-temporal signal representations (architecture, filter shapes, data augmentation, etc.)? For example, ...
0
votes
1answer
17 views

How to analyse the results of cross-validation do determine overfitting

I performed k-fold CV and measured the resulting average error (RMSE) for each fold. This was done with 5 folds, and 4 of the measurements gave similar errors (between 10% and 12%), but one of the ...
2
votes
2answers
30 views

When using Scikit Learn Grid Search, why are my train and cv scores high, but my test score is a lot lower?

I'm using scikit learn to run some models, and am very confused as to why my test score is so much lower than my cv score and my train score. At the start, I do a 80-20 train-test split. On the train ...
-1
votes
1answer
9 views

training accuracy greater than validation accuracy

The problem that I'm facing is that the training accuracy of my model is way higher than the validation accuracy, were talking about an approximate value of 0.2. ...
0
votes
2answers
75 views

How to handle Overfitting

I am working on machine learning classification problem with two classes (0/1). I would like to build a prediction model. The problem is that I have a small dataSet of ...
1
vote
1answer
36 views

Is this overfitting?

I read about the validations curves, and the following plot is similar to overfit, but in this case, the validation curve doesn't' growth again. So is this overfit? why? Thanks
0
votes
1answer
58 views

How to train a keras model on both original and augmented data from ImageDataGenerator?

I have a dataset that contains about 87000 images in a directory, with each class in a separate subfolder. I've tried the class ImageDataGenerator() and the ...
0
votes
1answer
31 views

Overfitting in Huggingface's TFBertForSequenceClassification

I'm using Huggingface's TFBertForSequenceClassification for multilabel tweets classification. During training the model archives good accuracy, but the validation accuracy is poor. I've tried to solve ...
3
votes
2answers
41 views

Mathematically prove why sparsity leads to model overfitting

With respect to the stackoverflow post here: https://stackoverflow.com/a/59566478/9130959 I can't quite get why the logic stands: when # features increases, the hypothesis space is expanded, leading ...
1
vote
1answer
33 views

Strange Neural Network overfitting

I'm experiencing a very strange behavior in training the following NN model for multiclass classification: ...
0
votes
0answers
27 views

XGBOOST/lLightgbm over-fitting despite no indication in cross-validation test scores?

We currently work on a project where we aim to identify a set of predictors that may influence the risk of a relatively rare outcome. We are using a semi-large clinical dataset, with data on nearly ...
0
votes
1answer
25 views

Validation accuracy greater than training accuracy in cnn

I've splitted my training set in the ratio 80:20 and have developed cnn model with a dropout of 0.5. I'm getting an accuracy of 98%. But the validation accuracy stays greater than training accuracy. ...
1
vote
1answer
8 views

Tuning SVM C parameter

I would like to ask for help regarding my model. I have a dataset of preprocessed images and I performed a binary classification with SVM on Python. I tuned the value of the c parameter from 0.001 to ...
5
votes
3answers
132 views

Disparity between training and testing errors with deep learning: the bias-variance tradeoff and model selection

I am developing a convolutional neural network and have a dataset with 13,000 datapoints that is split 80%/10%/10% train/validation/test. In tuning the model architecture, I found the following, after ...
1
vote
0answers
12 views

clustering more than optimal k and Overfitting in k-means

In my data by using elbow method. i got optimal k to be 3. but , i clustered them into 5 clusters.and the patterns in the cluster are as i wanted them . But, does using k more than optimal k decreases ...
0
votes
1answer
14 views

How to prevent neural network from overfitting on small subset of features

I'm trying to predict the win probability for a team in a basketball game using a neural network with a single sigmoid output. The input layer consists of a one-hot representation of the players, ...
7
votes
2answers
619 views

Is over fitting okay if test accuracy is high enough? [duplicate]

I am trying to build a binary classifier. I have tried deep neural networks with various different structures and parameters and I was not able to get anything better than ...
0
votes
0answers
25 views

Why does the LSTM overfit all the time

I have a time series prediction problem from building an LSTM. My code: ...
2
votes
1answer
21 views

How would you - on-the-fly - prevent a neural network from overfitting using a Keras callback?

I have a neural network that starts to overfit in that the validation loss begins to increase while the training loss stays ~ flat with epochs. Is there a generic algorithm - obvious or otherwise, ...
1
vote
0answers
28 views

Removing duplicate records before training

I am currently working on a project classifying text into classes. The specific problem is classifying job titles into various industry codes. For example "McDonalds Employee" might get classified to ...
0
votes
1answer
19 views

Amount of data needed for deep learning vs support vector machine

I often read about the fact, that the amount of data to train and get a generalizing model for a deep learning algorithm is much higher in comparison, e.g. to a support vector machine. It makes sense, ...
1
vote
2answers
26 views

Is this over-fitting or something else?

I recently put together an entry for the House Prices Kaggle competition for beginners. I decided to try my hand at understanding and using XGBoost. I split Kaggle's 'training' data into 'training' ...
0
votes
2answers
35 views

Weird overfitting in linear regression

This is really weird, I have a real simple test dataset and built a really simple linear model on it: ...
-1
votes
1answer
17 views

How does regularization help?

What is the effect of regularization on the value of parameters/weights? How does adding a regularization term in the cost function(J) and gradients help? Doesn't adding something increase the cost ...
1
vote
0answers
12 views

Oversampling for regression for data grouped in clusters

I am dealing with a regression problem in which I want to predict the upcoming value of a time-dependent variable by using the previous values of other variables (not including the output variable ...
1
vote
1answer
12 views

How can I tell whether my Random-Forest model is overfitting?

I was trying to generate predictions for Iris species using the UCI Machine Learning Iris dataset. I used a RandomForestClassifier with GridSearchCV and calculated the mean absolute error. However, ...
0
votes
0answers
53 views

None of the known overfitting prevention techniques works for me, according to learning curves

I am working on HTRU2 dataset to evaluate classification models. Even though I obtain good results in terms of accuracy-MSE: I have an overfitting problem according to the learning curves below. In ...
1
vote
1answer
75 views

Interpretation for test score , training score and validation score in machine learning?

Interpretation for test score , training score and validation score ? what they actually tell us? What's an acceptable difference between cross test score , validation score and test score? If ...
0
votes
0answers
25 views

Self-driving AI model starts overfitting

I'm trying to make a self-driving AI that can drive around in GTA-San Andreas by following sentdex's videos on making a self-driving AI for GTA-V but my model always starts to overfit after 4 epochs ...
0
votes
1answer
28 views

Use all available data to build Logistic Regression model [duplicate]

Using K-Fold, I chose to use Logistic Regression for a project of mine. I made it learn on my X_train (80% of data), and tested it on my X_test, with good results. My question is : now that I need ...
2
votes
2answers
35 views

Over fitting and association with regularization

Heard and read lot about regularization helps in reducing over fitting. But I'm not sure how exactly regularization works in reducing over fitting issue and whats the maths behind it? Appreciate if ...
1
vote
0answers
22 views

How to deal with training set that overfits very easily

I have a dataset consisting of 72 one-hot encoded (thus binary) features and 2.5K training examples. With this I am trying to solve a 10-class classification problem. My main problem is that no ...
0
votes
1answer
40 views

Should bias updates be porportional to overfitting?

According to questions on the internet, the bias is a learnable parameter, and there are different solutions to updating it, but I failed to find a concise methodology of correctly updating biases ...
2
votes
0answers
50 views

Bias Formula in Machine Learning expanded using ground truth

Why is Bias calculated for $f(x)$? Shouldn't it be calculated for $Y$ (which is $f(x)$ + Noise $\epsilon$)? We are fitting our model to $Y$, So shouldn't we be calculating bias wrt to $Y$? Also, I ...
1
vote
1answer
34 views

Training on skewed dataset

I have a problem of multi class classification and I'm using a simple 2-Layer Bi-directional LSTM with keras. The model in a simple form: ...
2
votes
1answer
29 views

Overfitting with sklearn pipeline - reasons why?

So.... I've been playing around with this for FAR TOOO LONG now and I really need some advice. Most people on kaggle concat training and testing set TOGETHER and then pre scale the data, this seems ...
1
vote
1answer
25 views

Using standard deviation as a metric for model selection

I'm really getting stuck with overfitting and I'm trying all I can to reduce it. I want't to write a metric to help score models in a cv loop. I'm using 10x5 folds and still getting out of sample ...
4
votes
1answer
190 views

ROC AUC score is much less than average cross validation score

Using Lending club Dataset to find the propability of default. I am using hyperopt library to fine tune hyper parameter for an XGBclassifier and trying to maximize the ROC AUC score. I am also using ...
0
votes
0answers
10 views

How to select checkpoint for model evaluation?

I have trained a deep convolutional neural network for image similarity classification. The network returns whether the images are the same or different. I trained the network for 20 epochs and save ...
3
votes
2answers
819 views

How to split data into 3 parts in Python - training(70%), validation(15%) and test(15%) and each part have similar target rate?

I'm working on a company project which I will need to do data partition into 3 parts - Train, Validation, and Test(holdout). Does anyone know how I can split the data into 3 parts above and each ...
4
votes
2answers
66 views

over-fitting with good enough test accuracy

Let's make things simple. Imagine an underdetermined linear system with $N$ samples and $p$ features $(N<p)$. Let's say I found one of the possible (among many) solutions of such systems and ...
1
vote
2answers
25 views

Will oversampling help with generalization (small imbalanced dataset)?

I have an imbalanced dataset (2:1 ratio) with about 60 patients and 80 features. I performed Recursive Feature Elimination (RFE) and stratified cross validation to reduce the features to 15 and I get ...
1
vote
0answers
40 views

How to interpret training results

I am working on an image similarity network. I have around 90,000 pairs of images contain an equal number of positive and negative samples. For learning the similarity between image pairs, I used the ...
31
votes
8answers
7k views

What would I prefer - an over-fitted model or a less accurate model?

Let's say we have two models trained. And let's say we are looking for good accuracy. The first has an accuracy of 100% on training set and 84% on test set. Clearly over-fitted. The second has an ...
2
votes
1answer
40 views

reduction of model accuracy while using PCA for a regression problem

I am trying to build a prection problem to predict the fare of flights. My data set has several catergorical variables like class,hour,day of week, day of month, month of year etc. I am using multiple ...
1
vote
2answers
169 views

What to choose: an overfit model with higher evaluation score or a non-overfit model with lower one?

For lack of a better term, overfit here means a higher discrepancy between train and validation score and non-overfit means a lower discrepancy. This "dilemma" just showed in neural network model I'...
2
votes
3answers
72 views

Is over-fitting a matter of features engineering or a matter of parameters set?

I am using sklearn package to make models. I tried randomly to set some paramater to a sklearn.ensemble.RandomForestClassifier ...
1
vote
3answers
151 views

Will the MAE of testing data always be higher than MAE of training data?

On the Kaggle Course Page the chart below shows that MAE of testing data is always higher than MAE of training data. Why is this the case? Is it only limited to DecisionTreeRegressor model? Or the ...