Questions tagged [overfitting]

Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

Difference between Validation Error on Learning Curve and Validation Error Calculation in Machine Learning Model

I am encountering a problem where the validation error I see on the learning curve of my machine learning model is different from the validation error I calculate using the mean squared error function....
0 votes
0 answers
6 views

Is there any concern for a pretrained model to overfitting to a fine-tuning task that has overlapping pretraining and training data?

Let's say my language model is pretrained on a general text corpus, and I want to use it for some specific downstream task that has it's datasets also included in the general corpus, is there any ...
  • 1
0 votes
1 answer
46 views

Can feature engineering avoid overfitting?

Can feature engineering avoid overfitting? If yes, are there any relevant papers that state this?
0 votes
0 answers
10 views

why by adding additional information as number of sequence on dataset can avoid overfitting

I am developing a regression model to analyze walking styles. The dataset I am using to build the model is from 2 different sources, let's call them dataset A and dataset B. Dataset A has a shape of (...
1 vote
1 answer
46 views

Imbalanced performance metrics in binary classification

I am developing a binary classification model using sklearn pipeline for preprocessing and a soft voting classifier (Adaboost and Extratrees with 50 estimators). The dataset (3 million rows) contains ...
0 votes
0 answers
25 views

Is this overfitting? (generative model)

I am working with a generative method, and the network seems to perform well on training data and slightly less well on test data, but the generated data is somehow significantly worse than either of ...
  • 31
0 votes
1 answer
23 views

Can I use macro recall to check if my RF model is overfitting?

I have a dataset with 837377 observations (51% to train, 25% to validation and 24% to test) and 19 features. I calculated the recall score using average macro for train, validation and test and ...
0 votes
0 answers
18 views

Transformer doesn't generalize on a time-series data

Data 120 patients. Each patient has 5.7556e+06 samples on average, each sample consists of 5 features stored as a continuous high-frequency (1000Hz) time series. Labels are 13 discrete classes ...
1 vote
1 answer
18 views

Underfitting and perfomance metrics in unsupervised methods

My question is simple and yet quite hard to find an answer to. In an unsupervised method, for example, when you have to reconstruct an input, how can you tell if your loss is good enough? Generally, ...
1 vote
1 answer
39 views

How to curve fit, Z variable dependent on X and Y?

I'm trying to find the function for this visualization: I would like to get feedback if I'm taking the right approach. My approach: These data points are created by a person. They are two ...
  • 111
0 votes
1 answer
32 views

Normalization / Overfitting Issues

I have a dataset with 608 inputs and I'm trying to output a single 1 or 0 result. My validation data has 69.12% 0's. When ran, my model always returns 69.12% accuracy, presumably because it's "...
0 votes
1 answer
9 views

Bad performance with CNN for basic image classification task

how are you doing? I'm playing around with CNN in FastAI. My model with 2 millions parameters only has around 80% accuracy. I also tried with Data normalization, Batch normalization, Label smoothing, ...
0 votes
0 answers
17 views

Cross sell model

I am interested in building a "cross-sell model" based on the current customer base. My main concern is that if I train my model using the current customer base while also attempting to ...
  • 1
0 votes
1 answer
27 views

A question about overfitting and SMOTE

So I understand that overfitting is when you have for example a good accuracy for the training dataset and bad one for the testing dataset, but why would I even check the accuracy for the training ...
  • 57
1 vote
1 answer
40 views

Interpreting Learning Curves of models

I need some help to understand if the models are overfitting and which of these we can consider "the best". On the internet i only find simple examples with learning curves but in these ...
  • 41
1 vote
0 answers
29 views

Why is my segmentation model not returning a heat map?

I have implemented two CNN architectures to perform segmentations on medical images: the classic UNet and a modified version called the Attention UNet. I have been training the models on roughly 50,...
  • 11
0 votes
2 answers
81 views

Relation between "underfitting" vs "high bias and low variance"

What is the exact relation between "underfitting" vs "high bias and low variance". They seem to be tightly related concepts but still 2 distinct things. What is the exact relation ...
  • 254
0 votes
0 answers
23 views

Bias-Variance Bulls-Eye Diagram: high variance and high bias

Often bias/variance trade-off is explained by a Bulls eye diagram. I like the explanation in the linked webpage but it doesn't answer the question how a model that has high variance and high bias ...
  • 254
0 votes
0 answers
17 views

Overfitting followed by well-fitting with large number of epochs

A few months ago I found an article it was explaining that some experiments had shown some networks would start well-fitting after overfitting happened in higher epochs. e.g. overfitting starts at ...
  • 101
1 vote
1 answer
80 views

Is there a point in hyperparameter tuning for Random Forests?

I have a binary classification task with substantial class imbalance (99% negative - 1% positive). I want to developed a Random Forest model to make prediction, and after establishing a baseline (with ...
  • 11
0 votes
0 answers
19 views

Interpreting high test accuracy

I am trying to build a classification xgboost model at work, and I'm potentially facing overfitting issue that I have never seen before. My training sample size is 320,000 X 718 and testing sample is ...
0 votes
0 answers
7 views

What inference can we make from this deep learning model's loss plot?

what reasons caused this type of output, simple overfit and underfit concepts does not apply here right, do they? Does local optimal points and learning rate parameter to the optimizers have an impact ...
0 votes
0 answers
17 views

My overfitted decision tree regressor gives better result than pre-pruned tree?

I create a decision tree regressor without giving any parameters. The resulting tree has 6255 leaf nodes (out of 6348 entries of train set) and depth of 39. Most probably it has overfitted. But its ...
0 votes
1 answer
121 views

Overfitting problem with small model

I have an encoder-decoder architecture where I have used top 3 layers of Swin Transformer and few convolutional layer. I tried different approach: i. Training the Transformer layers as well, on doing ...
1 vote
1 answer
51 views

Why is my model overfitting?

I am building a classification model based on some machine performance data. Unfortunately for me, it seems to over-fit no matter what I change. The dataset is quite large so I'll share the final ...
2 votes
0 answers
34 views

High overfitting, but test metrics are higher

Ml models must strike a balance between predictive power and generalization power. Therefore, I split the data into train/test and calculate metrics on both. Often I see instructions in someone else's ...
  • 376
0 votes
0 answers
8 views

Facial expression recognition with rsenet not giving good validation accuracy

I am using keras and my dataset is highly imbalanced. I am using resnet50 to train my model but its giving me 61% validation accuracy and 99% training accuracy. Its clear overfitting can u please help ...
0 votes
1 answer
55 views

Which metrics that can be use to know "overfitting" model"?

Hello everyone i'm new to data science world. So i want to know if my model is overfitting. Usually i'm comparing training accuracy and testing accuracy. But on some reference many people using ...
0 votes
1 answer
73 views

Binary classification performance difference between 0 and 1 class

I have trained a binary Random Forest classifier on a dataset containing 7M rows. I also set aside a holdout validation set of 1M rows that the training pipeline never sees. The dataset consists of ...
2 votes
0 answers
21 views

Discussion about modern deep learning training strategies

Previously I have put a lot of effort into training networks appropriately. However, talking to colleagues, a lot of the things I did may be redundant due to novel optimizers and the theory of deep ...
0 votes
1 answer
85 views

Bounding box regression without a classification task

I am using PyTorch to create a model that detects certain objects in an image. I framed my problem as a regression on bounding boxes, without any classification task whatsoever. The reasoning behind ...
0 votes
1 answer
35 views

Is there a way I can double the punishment when model mis-classing to a specific class?

As the title I asked. For example: a model that predicts the probability of a stock price rising/falling. Let's say this is a triple-classification problem. If it predicts "RISING", while ...
  • 101
4 votes
0 answers
54 views

Does ROC AUC different between crossval and test set indicate overfitting or other problem?

I am training a composite model (XGBoost, Linear Regression, and RandomForest) to predict injured people probability. Well, the results of cross-validation with 5 folds. Well, I can see any problem ...
2 votes
2 answers
465 views

SGD performing better than Adam in Random minority oversampling, I don't know what is the reason. Help

So my dataset image before and after balancing looks like this: But when I train with Adam(0.0001) and SGD(0.0001), the results are very different. Why? What is going on under the hood? This is ...
0 votes
1 answer
20 views

Where should I stop training if I want to bag models

Let's say I have a clear case of overfitting where my loss curves look like this (x axis are iterations): Now I would like to try bagging to reduce the variance, where should I stop models training? ...
  • 111
1 vote
1 answer
30 views

Evaluate best model [duplicate]

Let's assume I have 2 models Model 1: Train Accuracy = 92.4% Validation Accuracy = 37.6% Test Accuracy = 35.3% Model 2: Train Accuracy = 37.0% Validation Accuracy = 34.2% Test Accuracy = 34.1% ...
4 votes
3 answers
146 views

Timing of applying random oversampling on the dataset

I tried to learn classification using machine learning algorithms. I went through Breast Cancer - EDA, Balancing and ML the notebook. In this notebook ...
  • 359
0 votes
0 answers
27 views

Plot overfit of multi variable input model

In my machine learning project, I have created a linear regression model that has an input of 4 variables and returns output variable. Before adapting and processing the data, my model was overfitting,...
0 votes
0 answers
10 views

Need help diagnosing a training curve for LSTM-network

I am doing time series prediction using and LSTM-network. The dataset is divided into a training, test and validation set. The LSTM-model structure (number of neurons and layers), learning rate, batch ...
0 votes
0 answers
28 views

Is it a good idea to test the robustness of a Neural Network on a linear relation?

Just to give you more context, I'm currently working on a finance project relying on neural network. I'm principally using Neural Network to achieve regression task. So my neural network aims to ...
0 votes
0 answers
27 views

Why my neural network model is not able to learn a simple linear function?

I'm trying, using a neural network, to predict a simple relation $ f(x) = 1-x $. I write my function this way: $f(x) = 1 - x^{+} + (-x)^{+} $ or in a more data-scientistic way: $f(x)= 1-Relu(x)+Relu(-...
1 vote
2 answers
43 views

Can you use a trained image segmentation model to label more training data for itself?

Labeling images for semantic segmentation can be expensive. Is it viable to train a model (such as Unet) to a good accuracy and then use this model to label more images to be used as further training ...
0 votes
1 answer
252 views

How is the accuracy at the beginning of an epoch higher than that at the end of the previous one?

Below is a toy example of a CNN that I am trying out. As is observed, the accuracy at the beginning of the first epoch is at 84% and it increases to 96% by the end. With my understanding of ...
0 votes
0 answers
19 views

How to Predict Binary Classification problem having less dominant Features

I have following dataset. Total 31 columns including Target. Target column has value of either 1 or 0. This is balanced dataset. All 30 Feature columns also have value of either 1 or 0. All these 30 ...
  • 101
0 votes
1 answer
80 views

Spot Logistic Regression Training Error

My friend gave me this puzzle awhile ago and I've never figured it out. ...
1 vote
1 answer
50 views

Balancing between accuracy and model overfitting

I have a dataset and I have built an XGBClassifier model from it. Without hyperparameter tuning, the model performs fairly well in training but on test which have some signs of overfitting (train ...
1 vote
0 answers
14 views

State-of-the-art techniques for regularizing Neural Networks?

For regularizing neural networks, I'm familiar with drop-out and l2/l1 regularization, which were the biggest players in the late 2010's. Have any significant/strong competitors risen up since then?
  • 111
0 votes
0 answers
39 views

Batch size to avoid overfitting

I have written code for binary text classification using XLM-RoBERTaForSequenceClassification. My train_dataset is made up over 10.000 data. For training I have used a batch size=32. The text hasn't ...
0 votes
0 answers
28 views

Is this XGBoost model tending to overfit?

Here is the list of hyperparameters that I used: ...
0 votes
1 answer
37 views

Overfitting problem: high accurance and low accurancy validation for image classification

I want to define a model to predict 3 categories of images. I'm learnong on the field :-) I've 1500 images (500 for each category) in 3 directories. I've read in this blog many suggestions: use a ...

1
2 3 4 5
7