Questions tagged [generalization]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
33 views

LGBM model predicting only single class on unseen data!

I have built a LightGBM based machine learning model on data of molecules of two classes. The distribution is as follows. Class 0 has 5933 data points and class 1 has 4696. The train test accuracy I ...
  • 37
0 votes
1 answer
26 views

About neural network ability to generalize

I had this question during an interview that I wasn't able to answer, even after researching on the internet. Which of the following can affect an artificial neural network’s ability to generalize??? ...
0 votes
0 answers
18 views

Are my classification results like accuracy, precision, recall etc significant and valid for general data?

So I have this data let's say of size (2000,11), and I want to do perform a binary classification based on these eleven features. There is a class Imbalance between ...
  • 11
2 votes
1 answer
189 views

Which type of models generalize better, generative or discriminative models?

In NLP, which type of models (generative or discriminative) is more sensitive to the amount of data to generalize better? references? This is related to the way those two types capture the data ...
  • 242
0 votes
0 answers
60 views

How may I find the formula that produces greatest distance between data points using machine learning?

I currently have a data set shown below: ...
  • 101
0 votes
0 answers
22 views

Can there be scenarios where an overfitted model in machine learning cannot be generalized?

Is it always possible to generalize an overfitted model? I know there are ways to handle overfitting, but can there be scenarios where overfitting cannot be handled in machine learning?
1 vote
2 answers
247 views

Graph Neural Network fails at generalizing on unseen graph topologies

I'm using PytorchGeometric to train a graph convolutional network for regression over nodes problem (the graph models physical phenomena in the network of sensors; the network of sensors is actually ...
  • 11
1 vote
0 answers
12 views

Generalization error problem on training set

Training data: $\mathcal {T} =\{(2,1),(3,2),(4,6),(0,0),(1,1)\}$ you already computed a predictor for the output using linear regression by least squares, where you used the first 3 samples as ...
2 votes
1 answer
184 views

Does convergence equal learning in Deep Q-learning?

In my current research project I'm using the Deep Q-learning algorithm. The setup is as follows: I'm training the model (using Deep Q-learning) on a static dataset made up of experiences extracted ...
  • 81
1 vote
3 answers
77 views

Is my model underfitting?

Model: ...
  • 23
1 vote
0 answers
25 views

Reducing High generalization-error on industrial fault data

I have a industrial dataset containing labeled machine data for fault classification(3 classes: 1 ok, 2 for faults). The problem is that i have less (~16) different machines, thus iam currently having ...
  • 41
8 votes
2 answers
2k views

High accuracy on test-set, what could go wrong?

You are given a pre-trained binary ML classification model with 99% accuracy on the test-set (assume the customer required 95% and that the test-set is balanced). We would like to deploy our model in ...
2 votes
1 answer
177 views

Multilabel Classification - Overfitting?

My task is the following: To input drug combinations and output renal failure-related symptoms from the drug combinations. Both the drug combinations and renal-failure related symptoms are represented ...
  • 1,321
-1 votes
1 answer
119 views

Why when my local cv of loss decreases, my leaderboard's loss increases?

I got a cv log_loss of 0.3025410331400577 when using 4-fold cross-validation and my leaderboard (with 30% of test dataset) got 0.26514. I further did feature engineering and added some features to the ...
1 vote
1 answer
43 views

What is the form of data used for prediction with generalized stacking ensemble?

I am very confused as to how training data is split and on what data level 0 predictions are made when using generalized stacking. This question is similar to mine, but the answer is not sufficiently ...
1 vote
1 answer
118 views

Upper bound on size of sample set for decision trees

Say I have an instance space with 4 features and I know that a decision tree with 8 nodes can represent the target function I want to learn. I want to give an upper bound on the size of the sample set ...
0 votes
1 answer
914 views

how to shuffle the data for model.fit with custom data generator?

So trainfiles is a list that contains the files' directory and name e.g. ['../train/1.npy' , '../train/2.npy'] and then I create a dataset as shown in the middle of the code then I apply it to model ...
0 votes
2 answers
566 views

Compare Classification Performance in Datasets drawn from Different Populations

I've read some classics about comparison of ML Algorithms i.e. ...
  • 69
2 votes
1 answer
234 views

Why does this paper say that 0-1 loss is insensitive to scaling of weights in a neural network?

When discussing capacity control using norms of weights in a neural network,this paper says the following(see P4): Capacity control in terms of norm, when using a zero/one loss (i.e. counting ...
  • 121
1 vote
2 answers
85 views

Will oversampling help with generalization (small imbalanced dataset)?

I have an imbalanced dataset (2:1 ratio) with about 60 patients and 80 features. I performed Recursive Feature Elimination (RFE) and stratified cross validation to reduce the features to 15 and I get ...
0 votes
1 answer
66 views

Are there some research papers about text-to-set generation?

I have googled but find no results. Text-to-(word)set generation or sequence-to-(token)set generation. For example, input a text and then output the tags for this text: ...
  • 591
5 votes
1 answer
45 views

On design of the training set: conceptual question

I am curious to know how training data should be constructed so that it scales to examples that are not a part of the training data. For example, the problem that I am facing right now is in the ...
  • 521
1 vote
0 answers
19 views

How much can be learned from a dataset?

Is there a way to quantify or characterize some upper limits for how much one can "learn" from a dataset? I got puzzled, when the neural state machine got me thinking components of AGI and lead me ...
  • 21
1 vote
1 answer
438 views

Generalization of RNN/LSTM/GRU... model

Given a time-series prediction with a Recurrent Neural Network (doesn't matter if LSTM/GRU/...), a forecast might look like this: to_predict (orange) was fed to the model, predicted (purple) is the ...
  • 111
0 votes
1 answer
36 views

Is generalizing a model, then removing the generalization good for FFNNs?

If one is training a basic FFNN (Feed-Forward Neural Network), one would apply regularizations like dropout, l1, l2 and gaussian noise, so that the model is robust and gives better results for unseen ...
2 votes
1 answer
268 views

XGBoost: what to do when Kfold is not enough?

I have a dataset made of roughly 100 time-series and my final goal is to obtain a classification of each point (detection problem). To do so I have labels so I decided to use an XGB model to perform ...
0 votes
1 answer
206 views

How can I measure the reliability of the specificity of a model with very small train, test, and validation datasets?

Stats newbie here. I have a small dataset of 646 samples that I've trained a reasonably performant model on (~99% test and val accuracy). To complicate things a little bit, the classes are somewhat ...
  • 312
1 vote
0 answers
66 views

Network either overfits or underfits, but never generalizes - what to do?

I have a simple network with 1st level an LSTM, dropout, fully-connected and softmax layers; loss is cross-entropy (four classes, well balanced). Sequence length to LSTM is 172 samples, data is z-...
3 votes
1 answer
560 views

What is NLP technique to generalize manually created rules in text?

Let's say we have a free text containing key-value entities. Example: "... patient's tumour has width 6 cm and height 5 cm" Then an expert comes, marks it as important, thus we do have the rule for ...
  • 133
5 votes
1 answer
1k views

Training on accurate data versus noisy data

I have data currently available that is very accurate and I would like to train my classification methods on this set of clean data to learn the important markers for distinguishing between classes. ...
  • 185
0 votes
2 answers
197 views

Is a Neural Network with 20 times the number of input neurons (on hidden layers) guaranteed to overfit? When is this not so?

I'm aware of the problem of over-fitting. One way to describe it is your Neural Network learning your training data to a high accuracy and performing poorly (generalizing) on new data. Was wondering ...
0 votes
2 answers
629 views

Why is it bad to use the same test dataset over and over again?

I am following this Google's series: Machine Learning Crash Course. On the chapter about generalisation, they make the following statement: Good performance on the test set is a useful indicator of ...
1 vote
2 answers
269 views

How is the equation for the relation between prediction error, bias, and variance defined?

I'm reading this article Understanding the BiasVariance Tradeoff. It mentioned: If we denote the variable we are trying to predict as $Y$ and our covariates as $X$, we may assume that there is a ...
12 votes
2 answers
3k views

The differences between SVM and Logistic Regression

I am reading about SVM and I've faced to the point that non-kernelized SVMs are nothing more than linear separators. Therefore, ...
  • 5,714
5 votes
1 answer
1k views

The connection between optimization and generalization

Optimization algorithms such as gradient descent or particle swarm can find a minima in a function. On the other hand, learning methods such as back-prop define learning as an optimization problem ...