Questions tagged [overfitting]

Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.

Filter by
Sorted by
Tagged with
0 votes
0 answers
25 views

Binary classification performance difference between 0 and 1 class

I have trained a binary Random Forest classifier on a dataset containing 7M rows. I also set aside a holdout validation set of 1M rows that the training pipeline never sees. The dataset consists of ...
2 votes
0 answers
17 views

Discussion about modern deep learning training strategies

Previously I have put a lot of effort into training networks appropriately. However, talking to colleagues, a lot of the things I did may be redundant due to novel optimizers and the theory of deep ...
0 votes
1 answer
19 views

Bounding box regression without a classification task

I am using PyTorch to create a model that detects certain objects in an image. I framed my problem as a regression on bounding boxes, without any classification task whatsoever. The reasoning behind ...
0 votes
1 answer
28 views

Is there a way I can double the punishment when model mis-classing to a specific class?

As the title I asked. For example: a model that predicts the probability of a stock price rising/falling. Let's say this is a triple-classification problem. If it predicts "RISING", while ...
  • 101
3 votes
0 answers
46 views

Does ROC AUC different between crossval and test set indicate overfitting or other problem?

I am training a composite model (XGBoost, Linear Regression, and RandomForest) to predict injured people probability. Well, the results of cross-validation with 5 folds. Well, I can see any problem ...
3 votes
1 answer
126 views

SGD performing better than Adam in Random minority oversampling, I don't know what is the reason. Help

So my dataset image before and after balancing looks like this: But when I train with Adam(0.0001) and SGD(0.0001), the results are very different. Why? What is going on under the hood? This is ...
0 votes
1 answer
18 views

Where should I stop training if I want to bag models

Let's say I have a clear case of overfitting where my loss curves look like this (x axis are iterations): Now I would like to try bagging to reduce the variance, where should I stop models training? ...
  • 101
2 votes
1 answer
29 views

Evaluate best model [duplicate]

Let's assume I have 2 models Model 1: Train Accuracy = 92.4% Validation Accuracy = 37.6% Test Accuracy = 35.3% Model 2: Train Accuracy = 37.0% Validation Accuracy = 34.2% Test Accuracy = 34.1% ...
4 votes
3 answers
127 views

Timing of applying random oversampling on the dataset

I tried to learn classification using machine learning algorithms. I went through Breast Cancer - EDA, Balancing and ML the notebook. In this notebook ...
  • 309
0 votes
0 answers
14 views

Plot overfit of multi variable input model

In my machine learning project, I have created a linear regression model that has an input of 4 variables and returns output variable. Before adapting and processing the data, my model was overfitting,...
0 votes
0 answers
9 views

Need help diagnosing a training curve for LSTM-network

I am doing time series prediction using and LSTM-network. The dataset is divided into a training, test and validation set. The LSTM-model structure (number of neurons and layers), learning rate, batch ...
0 votes
0 answers
20 views

Is it a good idea to test the robustness of a Neural Network on a linear relation?

Just to give you more context, I'm currently working on a finance project relying on neural network. I'm principally using Neural Network to achieve regression task. So my neural network aims to ...
0 votes
0 answers
23 views

Why my neural network model is not able to learn a simple linear function?

I'm trying, using a neural network, to predict a simple relation $ f(x) = 1-x $. I write my function this way: $f(x) = 1 - x^{+} + (-x)^{+} $ or in a more data-scientistic way: $f(x)= 1-Relu(x)+Relu(-...
1 vote
2 answers
20 views

Can you use a trained image segmentation model to label more training data for itself?

Labeling images for semantic segmentation can be expensive. Is it viable to train a model (such as Unet) to a good accuracy and then use this model to label more images to be used as further training ...
0 votes
1 answer
29 views

How is the accuracy at the beginning of an epoch higher than that at the end of the previous one?

Below is a toy example of a CNN that I am trying out. As is observed, the accuracy at the beginning of the first epoch is at 84% and it increases to 96% by the end. With my understanding of ...
0 votes
0 answers
13 views

How to Predict Binary Classification problem having less dominant Features

I have following dataset. Total 31 columns including Target. Target column has value of either 1 or 0. This is balanced dataset. All 30 Feature columns also have value of either 1 or 0. All these 30 ...
  • 101
0 votes
1 answer
36 views

Spot Logistic Regression Training Error

My friend gave me this puzzle awhile ago and I've never figured it out. ...
1 vote
1 answer
41 views

Balancing between accuracy and model overfitting

I have a dataset and I have built an XGBClassifier model from it. Without hyperparameter tuning, the model performs fairly well in training but on test which have some signs of overfitting (train ...
1 vote
0 answers
13 views

State-of-the-art techniques for regularizing Neural Networks?

For regularizing neural networks, I'm familiar with drop-out and l2/l1 regularization, which were the biggest players in the late 2010's. Have any significant/strong competitors risen up since then?
  • 111
0 votes
0 answers
28 views

Batch size to avoid overfitting

I have written code for binary text classification using XLM-RoBERTaForSequenceClassification. My train_dataset is made up over 10.000 data. For training I have used a batch size=32. The text hasn't ...
0 votes
0 answers
16 views

Is this XGBoost model tending to overfit?

Here is the list of hyperparameters that I used: ...
0 votes
1 answer
36 views

Overfitting problem: high accurance and low accurancy validation for image classification

I want to define a model to predict 3 categories of images. I'm learnong on the field :-) I've 1500 images (500 for each category) in 3 directories. I've read in this blog many suggestions: use a ...
0 votes
0 answers
71 views

how to reduce overfitting and improve confusion matrix

I am trying to apply the following model on my data which is consists of (4030 samples as 5 classes) each sample is MFCC features which is extracted from an audio clip consisting of (20 second) and I ...
  • 27
0 votes
0 answers
129 views

How solved "ValueError: y should be a 1d array, got an array of shape () instead."?

...
0 votes
1 answer
72 views

Training loss decreasing while Validation loss is not decreasing

I am wondering why validation loss of this regression problem is not decreasing while I have implemented several methods such as making the model simpler, adding early stopping, various learning rates,...
1 vote
1 answer
74 views

How can i deal with this overfitting?

I trained my data over 40 epochs but got finally this shape. How can I deal with this problem? Please as I used 30.000 for training and 5000 for testing and ...
  • 47
0 votes
0 answers
25 views

Does eval loss decreasing slower than train loss indicate overfitting?

I am training a binary classifier using an efficientnetv2 model with a 1M image dataset where I do a 60/20/20 split. Does this graph mean that the model is over-fitting? I can see that the train loss ...
  • 1
0 votes
3 answers
84 views

Is my model overfitting ? Training Acc :93 % test accuracy 82%

I am using LGBM model for binary classification. After hyper-parameter tuning I get Training accuracy 0.9340 Test accuracy 0.8213 can I say my model is overfitting?...
  • 37
2 votes
2 answers
130 views

Low classification accuracy

I want to do a multi class classification with 6 classes. Whole dataset has 12750 and 56 features samples, so every class has 2125 samples. Before prediction I reduces amount of outliers by ...
  • 41
0 votes
1 answer
71 views

Overfitting CNN model - any relation to input image size?

If my CNN model is over-fitting despite trying all possible hyper parameter tuning, does it mean I must decrease/increase my input image size in the Imagadatagenarator?
  • 13
0 votes
0 answers
15 views

my k-fold validation is giving a lot of 100% in the concatenated confusion matrix, is it because of overfitting?

The confusion matrix is a concatenated one from a 5-fold stratified cross-validation of my data set. I used rbf kernel for the svm classifier. Is it telling me the classifier is overfitting? Plus when ...
  • 1
0 votes
0 answers
11 views

How to test if a curve is well described by an ellipse?

I have a set of data points in 2D, and I am trying to come up with some sort of statistical to determine if the points fall along an ellipse. My idea so far is to fit an ellipse to the points, take ...
  • 1
0 votes
0 answers
24 views

How do I choose the right parameters for just plain old simple standarddeviation?

I am evaluating different models that do binary classifications and basically generate trade signals. They make a prediction of either buy or sell for the next day. I look at 10 different underlying ...
  • 101
0 votes
1 answer
43 views

What causes explosion in MSE when training?

I (probably) well overfitted/overtrained a model. But I was just curious as to what might cause this type of behaviour. I carried on training (Epoch 1/50 is not the first epoch of training this model)....
  • 71
0 votes
1 answer
31 views

Why my models have a pretty high accuracy with a small training dataset?

I was wondering why my models (decision tree, svm, random forest) behave like that, with "high" accuracy on a small training dataset. Is it a sign of overfitting? The graph represents the ...
0 votes
1 answer
68 views

Assess overfitting - All model metrics or only specific metric?

I am working on a binary classification using random forest with 977 records with 77:23 class proportion I got the below performance in train and test data (AUC = 81) Train data Test data My metric ...
  • 2,337
1 vote
1 answer
22 views

Not perfect accuracy when overfitting

Given a dataset and a decision tree that can be as depth as it wants, if you train the tree with all the dataset and then you test it against the same dataset and you get an accuracy that is not 100%, ...
0 votes
0 answers
30 views

How should I improve my CNN binary classification model from overfitting and underfitting [duplicate]

I am trying to do the cats & dogs classification problem, the problem is that my model is overfitting and I have tried all the techniques I know in order to solve but nothing is working such as ...
2 votes
1 answer
92 views

Is data leakage giving me misleading results? Independent test set says no!

TLDR: I evaluated a classification model using 10-fold CV with data leakage in the training and test folds. The results were great. I then solved the data leakage and the results were garbage. I then ...
  • 21
1 vote
0 answers
42 views

Do monotonic constraints prevent an XGboost to capture non-linear relationships in the data?

I have trained an XGBoost model (for a binary classification problem) and I have tested two scenarios: Scenario 1 - No Monotonic Constrained applied In this case I get a Gini on the training sample of ...
0 votes
1 answer
9 views

Do we need to define model everytime we need to train data in LSTM?

Suppose if I have two datasets where 1st dataset is AAPL stock price and 2nd dataset is GOOGL stock price. Now if I define the model as ...
0 votes
0 answers
101 views

Model Overfitting in text classification how to solve?

This is my CNN model i am doing text classification on mental health social media data. the model is overfitting as validation loss is much greater than training loss. There are three columns(Text, ...
0 votes
2 answers
62 views

Linear Regression model underfitting

here is the source code of the model and the csv file. Using the csv file I have to apply linear regression Algorithm on it using "Sales" and "Profit". Train the model in such a ...
0 votes
0 answers
33 views

Overfitting? Is it ok, if I've met my desired threshold?

I've trained a lightgbm classification model, selected features, and tuned the hyperparameters all to obtain a model that appears to work well. When I've come to evaluate it on an out of bag selection ...
3 votes
1 answer
532 views

Which method is more suitable? overfitting of traning data or low accuracy?

Recently, I tested two methods after embedding in my data, using Keras. Convolution after embedding Maxpooling after embedding The first method's loss and validation loss are like, The second ...
3 votes
1 answer
40 views

Request: Confirmation on my understanding of overfitting and regularization concepts

Overfitted models tend to have largely different (some very high, some comparatively low) coefficients/weights for different feature values. So, this means the model (when drawn as graph) will have ...
1 vote
2 answers
61 views

How to make a model suffer from underfitting

I would like to show an example of my model when it is overfitting, and when it is underfitting. Now overfitting is pretty straight forward, just train on small data, and the model will remember the ...
0 votes
0 answers
72 views

Handling Overfitting(diverging val loss and train loss) in Conv-LSTM

This is related to an Educational Research Project I'm using to learn stuff so please be kind. I'm working on a video binary classification task. I have almost 1400 videos of both classes. I also ...
  • 1
0 votes
1 answer
36 views

What is the problem that causes overfitting in the code?

** ...
0 votes
0 answers
18 views

Validation accuracy have an almost good accuracy but loss function is high too

Based on my project, I find a little problem there with the statement like this. I want to make model with neural networks for text dataset. Then I use Pad Sequence for my text and Array for the ...
  • 13

1
2 3 4 5
7