Questions tagged [overfitting]
Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.
371
questions
0
votes
1
answer
48
views
0
votes
1
answer
50
views
Random Forest overfitting to unbalanced data set
I am working on an unbalanced classification problem. I have have 2000 points which are positive, and 6000 points as -ve (chosen randomly from 100k universe of -ve points universe). Although I have ~...
- 75
0
votes
0
answers
35
views
Can't overfit Transformer Encoder
In the below code I am trying to train a very simple Transformer Encoder model to basically do nothing with its input. Giving some arbitrary input vector x, the aim of the model is then to output that ...
0
votes
0
answers
21
views
How to reduce the overfitting in my CNN model?
I am new in this world want practice for create a convolutional neuronal network. A model convolutional for image classification. I want classificate women and men images.
Previous, I did a course by ...
- 1
0
votes
0
answers
19
views
overfitting or not
Hello so i'm building a classification model i train my on various models and these are the metrices so i want to know if ther's an overfitting or not
0
votes
1
answer
55
views
Training ResNet50 model for binary classification
I want to use ResNet50 model to perform binary classification on a dataset spectrogram dataset. In order to do that I had to make a couple of modifications to the model's architecture:
Modified the ...
0
votes
0
answers
25
views
Using Embedding For Regularization
Is using embeddings for regularization a valid practice?
My reasoning for that is that encoding training/tests datasets into smaller vectors would allow a smaller network with fewer parameters and ...
1
vote
2
answers
82
views
Xgboost model predicting extreme values for events and non-events | Overfitting
Extreme values are predicted by my trained xgboost classification model in BQML for both events (Y=1) and non-events (Y=0). For all event observations, the model calculates probability scores that ...
- 133
1
vote
0
answers
15
views
Avoid overfitting to noise by a noise penalty approach instead of early stopping?
I came across this article on deep learning for computational MRI and found an interesting sentence "However, early stopping has to be performed to not overfit to the noisy measurements." ...
- 11
0
votes
0
answers
36
views
Classification Threshold Optimization after GridSearchCV
In my machine learning problem I am using a CNN to classify images.
Since my dataset is imbalanced I want to perform classification probability threshold tuning so I can find the optimal balance ...
0
votes
0
answers
9
views
Test accuracy plateaus when increasing max_depth -> inf
I've built a Random Forest model that classifies into four categories based on around 10 input features.
To test the accuracy, I performed 5-fold stratified cross validation using the ...
0
votes
0
answers
23
views
I defined a subclass of torch.utils.data.IterableDataset, which line of the following loader could have caused the issue and why?
The current model is overfitting quickly, i.e., the training error is minimized in a few epochs while the validation error remains high. Suppose training data is sufficient, i.e. self.cohort is set ...
0
votes
1
answer
23
views
Challenges in Predicting Molecule Activity
I want to share a concern I have. I want to obtain a machine learning model that can predict whether a molecule exhibits biological activity. For this purpose, I have a set of molecules that do ...
0
votes
1
answer
30
views
How to compare test vs train model performance
When comparing the test vs train model performance to ensure no overfitting (e.g., using AUC ROC as an example), is it better to select the model with the largest test score, or the model with the ...
- 735
0
votes
0
answers
36
views
Learning curve - Why does the train learning curve is flat?
I implemented a model in which I use Random Forest as classifier and I wanted to plot the learning curves for both training and test sets to decide what to do next in order to improve my model.
...
- 13
0
votes
0
answers
45
views
The cost function gets stuck at 120 epochs
I did a neural network in c++ to recognize handwritten digits using the MNIST dataset without any neural network pre-existing libraries. My network has 784 inputs neuron (the pixel of the image), 100 ...
- 1
0
votes
1
answer
29
views
model.evaluate gives low results?
i have an image dataset and there are 6300 images with 5 classes . The features extracted and dataset reduced to 256 features. This dataset gives good results(%99) when tested ANN with Backpropagation(...
0
votes
0
answers
30
views
Prevent Overfitting in Transfer Learning with small data
I have built a feed forward neural network to predict heat pumps energy consumption. Now, i want to use this model as a domain for other heat pumps via transfer learning.
I want to simulate the case ...
- 1
0
votes
0
answers
12
views
Large Language model for regression on urls
I am trying to fit a BERT model for a URL regression task. I have a URL as a feature and I have to predict a metric M for it. Keeping a learning rate like $10^{-5}$, the model is overfitting in about ...
- 1
0
votes
0
answers
3
views
Is a predictor with high i formation value bad? Is there other way to cross check it?
So, I am preparing a dataset for an ML algorithm, but I have run into a problem - the thing is that around 23 of 96 predictors have got IV more than 0.5 (the lowest is 1.7) and I am curious if it is ...
0
votes
0
answers
43
views
Need insights in how to reduce overfitting with MLPClassifier
I am new to data science. Please bear with me as I ask this long question. I am trying to do Speech Emotion Recognition with MLPCLassifier on RAVDESS and Crema datasets. I am predicting only three ...
0
votes
1
answer
58
views
Evaluating overfitting in a logistic regression model
I have developed a logistic regression model for a classification problem and obtained an AUC (Area Under the Curve) score of approximately 0.9. The model was estimated by splitting the available data ...
- 1
0
votes
1
answer
47
views
why my accuracy and recall become higher in testing than training. How to interpret
Why in hidden layer 2 and 3 in neural networks scratch, the accuracy and recall I got low, but in testing the accuracy and recall become higher. In hidden layer 4 it's get weird when sampling strategy ...
- 1
0
votes
0
answers
8
views
LSTM, seq to classification, why training on balanced data set yields such a good result?
I am using LSTM to classify the origin of people's names.
The input data is not balanced over target classes, so I used oversampling to balance it.
Now, I defined a simple LSTM model as follows:
<...
0
votes
0
answers
21
views
Model returns near perfect PR-AUC score but other metrics seem fine. Is my model overfitting?
I am currently working on a very imbalanced dataset:
24 million transactions (rows of data)
30,000 fraudulent transactions (0.1% of total transactions)
The dataset is split via Year, into three sets ...
1
vote
0
answers
57
views
MobileNet validation loss not decreasing over time
I am trying to train a MobileNetV2 on a custom dataset, to image Classification task.
Cardinality is 864 images, split in 70%/20%/10%, balanced between the 3 different classes.
Weights are pre-loaded ...
- 11
0
votes
2
answers
84
views
Is it possible to overfit a simple single variable linear regression model?
I searched this question and the answer I got was about a general regression model, rather than a single variable, linear regression model. If you increase the number of variables, you could fit a ...
0
votes
0
answers
23
views
Final Model Training Problem - Overfitting
I am working on a CNN project for multiclass classification. I implemented hyperparameter optimization to find the most suitable model, during which I got a best accuracy of 97.38%. I then took this ...
- 23
0
votes
1
answer
76
views
Overfitting still exists using different techniques on voice classification
I have 986 voice signals which have been collected by our team. The data set includes 745 healthy and 150 unhealthy voice signals. I split the data into 70% training and 20% validation and 10% test (...
- 1
0
votes
1
answer
60
views
How do I know If my regression model is underfitting?
How do we evaluate the performance of a regression model with a certain RMSE given that a domain knowledge performance metric is not present?
Maybe MAPE is one way of comparing the performance of my ...
- 31
0
votes
1
answer
30
views
Overfitted model [duplicate]
A classic question with an unclear answer, is it better to have an overfitted model performing better on a Cross-Validation setting, or a non-overfitted model performing worse?
In this context, higher ...
- 133
0
votes
1
answer
39
views
Is my model overfitted?
I am using a naive bayes classifier to classify 20 newsgroup dataset. My accuracy on the training set is 97 and on the testing set is 89. Is my model overfitted?
If it is what steps can I take to ...
0
votes
1
answer
61
views
Difference between Validation Error on Learning Curve and Validation Error Calculation in Machine Learning Model
I am encountering a problem where the validation error I see on the learning curve of my machine learning model is different from the validation error I calculate using the mean squared error function....
1
vote
1
answer
182
views
Is there any concern for a pretrained model to overfitting to a fine-tuning task that has overlapping pretraining and training data?
Let's say my language model is pretrained on a general text corpus, and I want to use it for some specific downstream task that has it's datasets also included in the general corpus, is there any ...
- 11
1
vote
1
answer
223
views
Can feature engineering avoid overfitting?
Can feature engineering avoid overfitting? If yes, are there any relevant papers that state this?
- 31
0
votes
0
answers
18
views
why by adding additional information as number of sequence on dataset can avoid overfitting
I am developing a regression model to analyze walking styles. The dataset I am using to build the model is from 2 different sources, let's call them dataset A and dataset B. Dataset A has a shape of (...
- 31
1
vote
1
answer
58
views
Imbalanced performance metrics in binary classification
I am developing a binary classification model using sklearn pipeline for preprocessing and a soft voting classifier (Adaboost and Extratrees with 50 estimators). The dataset (3 million rows) contains ...
- 23
0
votes
0
answers
38
views
Is this overfitting? (generative model)
I am working with a generative method, and the network seems to perform well on training data and slightly less well on test data, but the generated data is somehow significantly worse than either of ...
- 31
0
votes
1
answer
33
views
Can I use macro recall to check if my RF model is overfitting?
I have a dataset with 837377 observations (51% to train, 25% to validation and 24% to test) and 19 features.
I calculated the recall score using average macro for train, validation and test and ...
0
votes
0
answers
128
views
Transformer doesn't generalize on a time-series data
Data
120 patients.
Each patient has 5.7556e+06 samples on average, each sample consists of 5 features stored as a continuous high-frequency (1000Hz) time series.
Labels are 13 discrete classes ...
- 13
1
vote
1
answer
20
views
Underfitting and perfomance metrics in unsupervised methods
My question is simple and yet quite hard to find an answer to. In an unsupervised method, for example, when you have to reconstruct an input, how can you tell if your loss is good enough? Generally, ...
- 11
1
vote
1
answer
201
views
How to curve fit, Z variable dependent on X and Y?
I'm trying to find the function for this visualization:
I would like to get feedback if I'm taking the right approach. My approach:
These data points are created by a person. They are two ...
- 111
0
votes
1
answer
44
views
Normalization / Overfitting Issues
I have a dataset with 608 inputs and I'm trying to output a single 1 or 0 result. My validation data has 69.12% 0's. When ran, my model always returns 69.12% accuracy, presumably because it's "...
- 3
0
votes
1
answer
26
views
Bad performance with CNN for basic image classification task
how are you doing?
I'm playing around with CNN in FastAI.
My model with 2 millions parameters only has around 80% accuracy. I also tried with Data normalization, Batch normalization, Label smoothing, ...
0
votes
0
answers
25
views
Cross sell model
I am interested in building a "cross-sell model" based on the current customer base.
My main concern is that if I train my model using the current customer base while also attempting to ...
- 1
0
votes
1
answer
45
views
A question about overfitting and SMOTE
So I understand that overfitting is when you have for example a good accuracy for the training dataset and bad one for the testing dataset, but why would I even check the accuracy for the training ...
- 89
1
vote
1
answer
42
views
Interpreting Learning Curves of models
I need some help to understand if the models are overfitting and which of these we can consider "the best". On the internet i only find simple examples with learning curves but in these ...
- 41
1
vote
0
answers
82
views
Why is my segmentation model not returning a heat map?
I have implemented two CNN architectures to perform segmentations on medical images: the classic UNet and a modified version called the Attention UNet. I have been training the models on roughly 50,...
- 11
2
votes
2
answers
2k
views
Relation between "underfitting" vs "high bias and low variance"
What is the exact relation between "underfitting" vs "high bias and low variance". They seem to be tightly related concepts but still 2 distinct things. What is the exact relation ...
- 294
1
vote
0
answers
70
views
Bias-Variance Bulls-Eye Diagram: high variance and high bias
Often bias/variance trade-off is explained by a Bulls eye diagram. I like the explanation in the linked webpage but it doesn't answer the question how a model that has high variance and high bias ...
- 294