Questions tagged [overfitting]

Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.

Filter by
Sorted by
Tagged with
0
votes
1answer
470 views

How to gauge overfit with MLPClassifier and cross_val_score?

I'm learning sklearn. When using MLPClassifier.fit() and MLPClassifier.predict() I would ...
1
vote
1answer
45 views

regularization error vs over fitting

I have a collected data from 50 unique blocks, and then merged data from 49 blocks into one data set, and saved the data from 1 block for testing purpose. I then split the merged data set from 49 ...
0
votes
1answer
52 views

Number of units for first layer in Keras Sequential Model

I have a huge CSV structured dataset. I'm feeding that dataset to a Keras Sequential Model. My question is, can my Model have number of units greater than the number of input features? At the moment, ...
0
votes
1answer
31 views

How many features do I select when doing feature selection for regression algorithms? Is R2 and RMSE good measures of success for overfitting?

Context: I'm currently crafting and comparing machine learning models to predict housing data. I have around 32000 data points, 42 features, and I'm predicting housing price. I'm comparing Random ...
1
vote
1answer
32 views

Derive features from test-set?

I have a dataset of choices (between A,B and C) done by certain users, and I want to train a neural network to predict the choices. I divide in train and test sets. An instance is formed by: [UserId, ...
10
votes
4answers
31k views

High model accuracy vs very low validation accuarcy

I'm building a sentiment analysis program in python using Keras Sequential model for deep learning my data is 20,000 tweets: positive tweets: 9152 tweets negative tweets: 10849 tweets I wrote a ...
0
votes
0answers
27 views

My Stacked LSTM seems to be doing worse than a shallower one

I started with a two layer LSTM (+ Dense Layers) and which was: ...
2
votes
2answers
2k views

Minimum number of samples to train XGBoost without overfitting

When using Neural Networks for image processing I learned a rule of thumb: to avoid overfitting, supply at least 10 training examples for every neuron. Is there a similar rule of thumb for classifiers ...
0
votes
0answers
15 views

Overfitting when there is an “elbow” with number of FP in results [closed]

I am facing a binary classification problem that is highly imbalanced. The model chosen for that is lightgbm classifier which handles the categorical features out of the box. I have reduced the number ...
1
vote
1answer
17 views

NGBoost and overfit - which model is used?

While training an NGBoost model I got: ...
3
votes
1answer
201 views

Convolution neural network with 11 million parameters unable to overfit on 100 image samples

I have been trying to do some sort of image enhancement on grayscale images. I have used both pixel wise loss and perceptual loss (perceptual loss uses classifier between 2 classes trained on the same ...
6
votes
0answers
68 views

XGBoost skews towards minority class

I have a dataset with 85k positive labels and 53k negative labels. For this use-case, I am trying to maximize my efforts to the negative class (accurately identify true negatives, and minimize false ...
7
votes
2answers
242 views

Dropping features after final evaluation on test data

Would you please let me know if I am committing a statistical or machine learning mal-practice in this procedure? I want to estimate meteorological variable y1 from ${x_1, ..., x_{10}}$ variables. I ...
2
votes
1answer
29 views

machine learning disasters

I am writing a research paper and I am looking for reliable sources that provide information on disasters of machine learning. Especially in the field of autonomous driving. Have there been any ...
0
votes
2answers
93 views

How to train a keras model on both original and augmented data from ImageDataGenerator?

I have a dataset that contains about 87000 images in a directory, with each class in a separate subfolder. I've tried the class ImageDataGenerator() and the ...
1
vote
1answer
44 views

Hyperparameter tunning for Random Forest- choose the best max depth

I'm trying to choose the best parameters for random forest model. For that goal I hae run my model in loop with only one parameter and each time I have changed the number for the parameter max depth. ...
4
votes
2answers
144 views

Trying to figure out which the training set is

Can someone help me on this one? As can be seen in the screenshot, it says the loss is 1/2. Where's that 1/2 coming from? How can I replace the values in the h(s) function? Source PDF
1
vote
1answer
31 views

Augmenting the validation set in Ensemble Model

I have 8 models which I have trained on 90% of my set (training set) and tracked its performance on the loss of the validation set (10% of the original set). I want to generate an ensemble model by ...
5
votes
1answer
12k views

Validation loss increases and validation accuracy decreases

I have an issue with my model. I'm trying to use the most basic Conv1D model to analyze review data and output a rating of 1-5 class, therefore the loss is categorical_crossentropy. Model structure is ...
3
votes
1answer
223 views

Determining whether a Machine Learning model is overfitted with regard to the stability of the features

I need to know how would I get to know if I have overfitted my Machine Learning model on the train data. The performance metric I have used is Logistic Loss. Does the stability of the features affect ...
1
vote
3answers
4k views

Over fitting in Transfer Learning with small dataset

I am using Transfer Learning to perform image classification. Base model used : Resnet50 using ImageNet dataset ...
0
votes
1answer
50 views

Python: How to test a RandomForest regression model for Overfitting?

I'm a beginner in this area so maybe I'm doing something wrong here. I'm using RandomForest for a regression model and wanted to see if my model is overfitting. Here is what I did: EDIT: I use ...
0
votes
0answers
24 views

Overfitting in imbalanced dataset

I am working on a dataset related to an insurance company and the objective is to predict if the insurance buyer will claim their travel insurance or not. Training data: https://raw.githubusercontent....
2
votes
1answer
35 views

Why Continous Variable Buckets Overfitting model

I have a continuous (high cardinal discrete) variable 'numInteractionPoints' in my dataset during training model - I binned this feature in order to avoid overffing , first top bar chart is from ...
0
votes
1answer
37 views

why should i do target encoding within cv loop?

i wish to use target encoding, using the category encoders sklearn library. I don't really understand why it is necessary to include this as a step in a sklearn pipeline WITHIN the cross validation ...
1
vote
2answers
40 views

Will oversampling help with generalization (small imbalanced dataset)?

I have an imbalanced dataset (2:1 ratio) with about 60 patients and 80 features. I performed Recursive Feature Elimination (RFE) and stratified cross validation to reduce the features to 15 and I get ...
0
votes
0answers
13 views

Strange behavior of CNN when forecasting time series

I have a time series containing 5 features. I tried to use LSTM to predict the next 112 periods in the series. However, I got very bad results. So I tried to use CNN. First, it did not work properly ...
7
votes
1answer
269 views

Who invented the concept of over-fitting?

I list the references that I found so far. Shortly, the first appearance of the term was in 1670, first appearance in in close meaning was in 1827, first appearance in a biological paper was in 1923 ...
0
votes
1answer
25 views

How to analyse the results of cross-validation do determine overfitting

I performed k-fold CV and measured the resulting average error (RMSE) for each fold. This was done with 5 folds, and 4 of the measurements gave similar errors (between 10% and 12%), but one of the ...
0
votes
0answers
71 views

Reduce overfitting in a CNN model

We are Data science students and we are building a CNN model to pneumonia classification (dataset: https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia ). We have applied a data augmentation ...
3
votes
2answers
568 views

Over-sampling: is my model over-fitting?

I would like to ask you some questions on how to consider (good or not) the following results: ...
0
votes
1answer
116 views

Correctly evaluate model with oversampling and cross-validation

I'm dealing with a classic case of dataset with binary imbalanced target (event 3%, non event 97%). My idea is to apply some sort of sampling (over/under, SMOTE etc.) to address the issue. As I see, ...
0
votes
1answer
34 views

Training Object Detection model on just 10 images

I am trying to train an object detection model using Mask-RCNN with Resnet50 as backbone. I am using the pre-trained models from PyTorch's Torchvision library. I have only 10 images that I can use to ...
4
votes
5answers
2k views

Why does my model produce too good to be true output?

I am trying to run a binary classification problem on people with diabetes and non-diabetes. For labeling my datasets, I followed a simple rule. If a person has T2DM...
0
votes
1answer
286 views

Overfitting in Huggingface's TFBertForSequenceClassification

I'm using Huggingface's TFBertForSequenceClassification for multilabel tweets classification. During training the model archives good accuracy, but the validation accuracy is poor. I've tried to solve ...
0
votes
1answer
34 views

Why use regularization?

In a linear model, regularization decreases the slope. Do we just assume that fitting a lin model on training data overfits by almost always creating a slope which is higher than it would be with ...
3
votes
2answers
105 views

Overfitted model produces similar AUC on test set, so which model do I go with?

I was trying to compare the effect of running GridSearchCV on a dataset which was oversampled prior and oversampled after the training folds are selected. The oversampling approach I used was random ...
0
votes
1answer
31 views

Why might my validation loss flatten out while my training loss continues to decrease?

In my effort to learn a bit more about data science I scraped some labeled data from the web and am trying to classify examples into one of three classes. I am running into a problem that, regardless ...
0
votes
0answers
27 views

SVM overfitting with consistent validation results

I have some imbalanced (1400 samples of which 250 are +ve) data for a binary classification problem and I am running an SVM grid search optimising for precision. I am trying 3,4,5,6,7,and 8 stratified ...
0
votes
1answer
37 views

Compare cross validation and test set results

I am having a hard time understanding the results of a cross validation test and a test run on a test set. First I made the following pipeline: ...
34
votes
6answers
7k views

What would I prefer - an over-fitted model or a less accurate model?

Let's say we have two models trained. And let's say we are looking for good accuracy. The first has an accuracy of 100% on training set and 84% on test set. Clearly over-fitted. The second has an ...
2
votes
1answer
55 views

Normal distribution and Random Forest

I have big table in dataframe (600k rows) which has y column (the variable I want to predict) and other 4 other columns that are the X. I have run RF regressor and I got score of 0.87 when I run it ...
2
votes
2answers
36 views

Why is large decision tree likely to overfit

My lecture slide told me that if we don't prune the regression tree, then the tree likely to over-fit. So, I wonder why would that happen? Is that because if the tree grows too large, we would end up ...
0
votes
1answer
32 views

Regularization hyperparam tuning during training

I have an idea for a regularization-hyperparam selection method, which I haven't encountered before and can't find on Google, but I'm sure someone has already tried it and I'm wondering what are the ...
0
votes
1answer
28 views

Amount of data needed for deep learning vs support vector machine

I often read about the fact, that the amount of data to train and get a generalizing model for a deep learning algorithm is much higher in comparison, e.g. to a support vector machine. It makes sense, ...
0
votes
1answer
41 views

How can I find if it is an overfitting problem?

I am new in Machine learning, and I want to detect emotions from the face. Preprocessing: I used equalizeHist to equalizes the histogram of grayscale images (JAFFE database with 213 images), in the ...
3
votes
5answers
358 views

How many ways are there to check model overfitting?

I am running xgboost on a regression classification problem where the model is predicting a score of how likely a gene is to cause a disease from 0-1. I try to avoid overfitting in all the ways I can ...
1
vote
0answers
10 views

model tuning by using loss curves

I have been practicing with the following dataset: http://archive.ics.uci.edu/ml/datasets/Concrete+Compressive+Strength for building a prediction model based on a MLP, but I have some doubts if the ...
1
vote
1answer
52 views

On which step should use SMOTE technique for over sampling?

I want to use SMOTE technique for over sampling but I don't know on which step on pre-processing I should use it. My preprocessing steps are: Missing values Removing Outliers Smoothing Data Should ...
5
votes
2answers
195 views

Why an increasing validation loss and validation accuracy signifies overfitting?

When I train a neural network, I observe an increasing validation loss, while at the same time, the validation accuracy is also increased. I have read explanations related to the phenomenon, and it ...

1
2 3 4 5