Questions tagged [overfitting]
Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.
350
questions
0
votes
0
answers
8
views
Difference between Validation Error on Learning Curve and Validation Error Calculation in Machine Learning Model
I am encountering a problem where the validation error I see on the learning curve of my machine learning model is different from the validation error I calculate using the mean squared error function....
0
votes
0
answers
6
views
Is there any concern for a pretrained model to overfitting to a fine-tuning task that has overlapping pretraining and training data?
Let's say my language model is pretrained on a general text corpus, and I want to use it for some specific downstream task that has it's datasets also included in the general corpus, is there any ...
- 1
0
votes
1
answer
46
views
Can feature engineering avoid overfitting?
Can feature engineering avoid overfitting? If yes, are there any relevant papers that state this?
- 21
0
votes
0
answers
10
views
why by adding additional information as number of sequence on dataset can avoid overfitting
I am developing a regression model to analyze walking styles. The dataset I am using to build the model is from 2 different sources, let's call them dataset A and dataset B. Dataset A has a shape of (...
- 21
1
vote
1
answer
46
views
Imbalanced performance metrics in binary classification
I am developing a binary classification model using sklearn pipeline for preprocessing and a soft voting classifier (Adaboost and Extratrees with 50 estimators). The dataset (3 million rows) contains ...
- 23
0
votes
0
answers
25
views
Is this overfitting? (generative model)
I am working with a generative method, and the network seems to perform well on training data and slightly less well on test data, but the generated data is somehow significantly worse than either of ...
- 31
0
votes
1
answer
23
views
Can I use macro recall to check if my RF model is overfitting?
I have a dataset with 837377 observations (51% to train, 25% to validation and 24% to test) and 19 features.
I calculated the recall score using average macro for train, validation and test and ...
0
votes
0
answers
18
views
Transformer doesn't generalize on a time-series data
Data
120 patients.
Each patient has 5.7556e+06 samples on average, each sample consists of 5 features stored as a continuous high-frequency (1000Hz) time series.
Labels are 13 discrete classes ...
- 13
1
vote
1
answer
18
views
Underfitting and perfomance metrics in unsupervised methods
My question is simple and yet quite hard to find an answer to. In an unsupervised method, for example, when you have to reconstruct an input, how can you tell if your loss is good enough? Generally, ...
- 11
1
vote
1
answer
39
views
How to curve fit, Z variable dependent on X and Y?
I'm trying to find the function for this visualization:
I would like to get feedback if I'm taking the right approach. My approach:
These data points are created by a person. They are two ...
- 111
0
votes
1
answer
32
views
Normalization / Overfitting Issues
I have a dataset with 608 inputs and I'm trying to output a single 1 or 0 result. My validation data has 69.12% 0's. When ran, my model always returns 69.12% accuracy, presumably because it's "...
- 3
0
votes
1
answer
9
views
Bad performance with CNN for basic image classification task
how are you doing?
I'm playing around with CNN in FastAI.
My model with 2 millions parameters only has around 80% accuracy. I also tried with Data normalization, Batch normalization, Label smoothing, ...
0
votes
0
answers
17
views
Cross sell model
I am interested in building a "cross-sell model" based on the current customer base.
My main concern is that if I train my model using the current customer base while also attempting to ...
- 1
0
votes
1
answer
27
views
A question about overfitting and SMOTE
So I understand that overfitting is when you have for example a good accuracy for the training dataset and bad one for the testing dataset, but why would I even check the accuracy for the training ...
- 57
1
vote
1
answer
40
views
Interpreting Learning Curves of models
I need some help to understand if the models are overfitting and which of these we can consider "the best". On the internet i only find simple examples with learning curves but in these ...
- 41
1
vote
0
answers
29
views
Why is my segmentation model not returning a heat map?
I have implemented two CNN architectures to perform segmentations on medical images: the classic UNet and a modified version called the Attention UNet. I have been training the models on roughly 50,...
- 11
0
votes
2
answers
81
views
Relation between "underfitting" vs "high bias and low variance"
What is the exact relation between "underfitting" vs "high bias and low variance". They seem to be tightly related concepts but still 2 distinct things. What is the exact relation ...
- 254
0
votes
0
answers
23
views
Bias-Variance Bulls-Eye Diagram: high variance and high bias
Often bias/variance trade-off is explained by a Bulls eye diagram. I like the explanation in the linked webpage but it doesn't answer the question how a model that has high variance and high bias ...
- 254
0
votes
0
answers
17
views
Overfitting followed by well-fitting with large number of epochs
A few months ago I found an article it was explaining that some experiments had shown some networks would start well-fitting after overfitting happened in higher epochs. e.g. overfitting starts at ...
- 101
1
vote
1
answer
80
views
Is there a point in hyperparameter tuning for Random Forests?
I have a binary classification task with substantial class imbalance (99% negative - 1% positive). I want to developed a Random Forest model to make prediction, and after establishing a baseline (with ...
- 11
0
votes
0
answers
19
views
Interpreting high test accuracy
I am trying to build a classification xgboost model at work, and I'm potentially facing overfitting issue that I have never seen before.
My training sample size is 320,000 X 718 and testing sample is ...
- 121
0
votes
0
answers
7
views
What inference can we make from this deep learning model's loss plot?
what reasons caused this type of output, simple overfit and underfit concepts does not apply here right, do they?
Does local optimal points and learning rate parameter to the optimizers have an impact ...
- 11
0
votes
0
answers
17
views
My overfitted decision tree regressor gives better result than pre-pruned tree?
I create a decision tree regressor without giving any parameters. The resulting tree has 6255 leaf nodes (out of 6348 entries of train set) and depth of 39. Most probably it has overfitted. But its ...
- 1
0
votes
1
answer
121
views
Overfitting problem with small model
I have an encoder-decoder architecture where I have used top 3 layers of Swin Transformer and few convolutional layer. I tried different approach:
i. Training the Transformer layers as well, on doing ...
- 109
1
vote
1
answer
51
views
Why is my model overfitting?
I am building a classification model based on some machine performance data. Unfortunately for me, it seems to over-fit no matter what I change. The dataset is quite large so I'll share the final ...
2
votes
0
answers
34
views
High overfitting, but test metrics are higher
Ml models must strike a balance between predictive power and generalization power. Therefore, I split the data into train/test and calculate metrics on both. Often I see instructions in someone else's ...
- 376
0
votes
0
answers
8
views
Facial expression recognition with rsenet not giving good validation accuracy
I am using keras and my dataset is highly imbalanced. I am using resnet50 to train my model but its giving me 61% validation accuracy and 99% training accuracy. Its clear overfitting can u please help ...
0
votes
1
answer
55
views
Which metrics that can be use to know "overfitting" model"?
Hello everyone i'm new to data science world.
So i want to know if my model is overfitting. Usually i'm comparing training accuracy and testing accuracy. But on some reference many people using ...
- 138
0
votes
1
answer
73
views
Binary classification performance difference between 0 and 1 class
I have trained a binary Random Forest classifier on a dataset containing 7M rows. I also set aside a holdout validation set of 1M rows that the training pipeline never sees. The dataset consists of ...
- 23
2
votes
0
answers
21
views
Discussion about modern deep learning training strategies
Previously I have put a lot of effort into training networks appropriately. However, talking to colleagues, a lot of the things I did may be redundant due to novel optimizers and the theory of deep ...
- 21
0
votes
1
answer
85
views
Bounding box regression without a classification task
I am using PyTorch to create a model that detects certain objects in an image. I framed my problem as a regression on bounding boxes, without any classification task whatsoever. The reasoning behind ...
- 1
0
votes
1
answer
35
views
Is there a way I can double the punishment when model mis-classing to a specific class?
As the title I asked.
For example: a model that predicts the probability of a stock price rising/falling. Let's say this is a triple-classification problem.
If it predicts "RISING", while ...
- 101
4
votes
0
answers
54
views
Does ROC AUC different between crossval and test set indicate overfitting or other problem?
I am training a composite model (XGBoost, Linear Regression, and RandomForest) to predict injured people probability. Well, the results of cross-validation with 5 folds. Well, I can see any problem ...
- 116
2
votes
2
answers
465
views
SGD performing better than Adam in Random minority oversampling, I don't know what is the reason. Help
So my dataset image before and after balancing looks like this:
But when I train with Adam(0.0001) and SGD(0.0001), the results are very different. Why? What is going on under the hood?
This is ...
- 21
0
votes
1
answer
20
views
Where should I stop training if I want to bag models
Let's say I have a clear case of overfitting where my loss curves look like this (x axis are iterations):
Now I would like to try bagging to reduce the variance, where should I stop models training?
...
- 111
1
vote
1
answer
30
views
Evaluate best model [duplicate]
Let's assume I have 2 models
Model 1:
Train Accuracy = 92.4%
Validation Accuracy = 37.6%
Test Accuracy = 35.3%
Model 2:
Train Accuracy = 37.0%
Validation Accuracy = 34.2%
Test Accuracy = 34.1%
...
4
votes
3
answers
146
views
Timing of applying random oversampling on the dataset
I tried to learn classification using machine learning algorithms. I went through Breast Cancer - EDA, Balancing and ML the notebook. In this notebook ...
- 359
0
votes
0
answers
27
views
Plot overfit of multi variable input model
In my machine learning project, I have created a linear regression model that has an input of 4 variables and returns output variable. Before adapting and processing the data, my model was overfitting,...
0
votes
0
answers
10
views
Need help diagnosing a training curve for LSTM-network
I am doing time series prediction using and LSTM-network. The dataset is divided into a training, test and validation set. The LSTM-model structure (number of neurons and layers), learning rate, batch ...
- 1
0
votes
0
answers
28
views
Is it a good idea to test the robustness of a Neural Network on a linear relation?
Just to give you more context, I'm currently working on a finance project relying on neural network. I'm principally using Neural Network to achieve regression task. So my neural network aims to ...
0
votes
0
answers
27
views
Why my neural network model is not able to learn a simple linear function?
I'm trying, using a neural network, to predict a simple relation $ f(x) = 1-x $.
I write my function this way: $f(x) = 1 - x^{+} + (-x)^{+} $ or in a more data-scientistic way: $f(x)= 1-Relu(x)+Relu(-...
1
vote
2
answers
43
views
Can you use a trained image segmentation model to label more training data for itself?
Labeling images for semantic segmentation can be expensive. Is it viable to train a model (such as Unet) to a good accuracy and then use this model to label more images to be used as further training ...
0
votes
1
answer
252
views
How is the accuracy at the beginning of an epoch higher than that at the end of the previous one?
Below is a toy example of a CNN that I am trying out. As is observed, the accuracy at the beginning of the first epoch is at 84% and it increases to 96% by the end.
With my understanding of ...
0
votes
0
answers
19
views
How to Predict Binary Classification problem having less dominant Features
I have following dataset.
Total 31 columns including Target.
Target column has value of either 1 or 0.
This is balanced dataset.
All 30 Feature columns also have value of either 1 or 0.
All these 30 ...
- 101
0
votes
1
answer
80
views
Spot Logistic Regression Training Error
My friend gave me this puzzle awhile ago and I've never figured it out.
...
1
vote
1
answer
50
views
Balancing between accuracy and model overfitting
I have a dataset and I have built an XGBClassifier model from it. Without hyperparameter tuning, the model performs fairly well in training but on test which have some signs of overfitting (train ...
- 161
1
vote
0
answers
14
views
State-of-the-art techniques for regularizing Neural Networks?
For regularizing neural networks, I'm familiar with drop-out and l2/l1 regularization, which were the biggest players in the late 2010's.
Have any significant/strong competitors risen up since then?
- 111
0
votes
0
answers
39
views
Batch size to avoid overfitting
I have written code for binary text classification using XLM-RoBERTaForSequenceClassification. My train_dataset is made up over 10.000 data. For training I have used a batch size=32. The text hasn't ...
- 1
0
votes
0
answers
28
views
Is this XGBoost model tending to overfit?
Here is the list of hyperparameters that I used:
...
0
votes
1
answer
37
views
Overfitting problem: high accurance and low accurancy validation for image classification
I want to define a model to predict 3 categories of images. I'm learnong on the field :-)
I've 1500 images (500 for each category) in 3 directories. I've read in this blog many suggestions:
use a ...
