Questions tagged [generalization]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
13 views

LGBM model predicting only single class on unseen data!

I have built a LightGBM based machine learning model on data of molecules of two classes. The distribution is as follows. Class 0 has 5933 data points and class 1 has 4696. The train test accuracy I ...
user avatar
  • 27
0 votes
1 answer
20 views

About neural network ability to generalize

I had this question during an interview that I wasn't able to answer, even after researching on the internet. Which of the following can affect an artificial neural network’s ability to generalize??? ...
user avatar
0 votes
0 answers
15 views

Are my classification results like accuracy, precision, recall etc significant and valid for general data?

So I have this data let's say of size (2000,11), and I want to do perform a binary classification based on these eleven features. There is a class Imbalance between ...
user avatar
  • 11
2 votes
1 answer
179 views

Which type of models generalize better, generative or discriminative models?

In NLP, which type of models (generative or discriminative) is more sensitive to the amount of data to generalize better? references? This is related to the way those two types capture the data ...
user avatar
  • 240
0 votes
0 answers
59 views

How may I find the formula that produces greatest distance between data points using machine learning?

I currently have a data set shown below: ...
user avatar
  • 101
0 votes
0 answers
14 views

Improved generalization by training on adversarial examples

This question is based on the following intuition: To my understanding adversarial attacks work, because the model is stuck in a local minima and the adversarial attack finds this with gradient ...
user avatar
0 votes
0 answers
21 views

Can there be scenarios where an overfitted model in machine learning cannot be generalized?

Is it always possible to generalize an overfitted model? I know there are ways to handle overfitting, but can there be scenarios where overfitting cannot be handled in machine learning?
user avatar
0 votes
0 answers
4 views

Should I generalize categorical features if the algorithm handles over-fitting well?

I'm referring to Kaggle feature creation exercise . The data frame contains a column(MSSubClass) that contains these unique values: ...
user avatar
0 votes
0 answers
8 views

Multiclass classification oob error

Im implementing a random forrest for a 6 class classification and witnessing a strange phenomenon. I have 10 percent of my set sectioned out as a pseudo validation set. Im training 50 percent of the ...
user avatar
1 vote
2 answers
187 views

Graph Neural Network fails at generalizing on unseen graph topologies

I'm using PytorchGeometric to train a graph convolutional network for regression over nodes problem (the graph models physical phenomena in the network of sensors; the network of sensors is actually ...
user avatar
  • 11
1 vote
0 answers
11 views

Generalization error problem on training set

Training data: $\mathcal {T} =\{(2,1),(3,2),(4,6),(0,0),(1,1)\}$ you already computed a predictor for the output using linear regression by least squares, where you used the first 3 samples as ...
user avatar
2 votes
1 answer
156 views

Does convergence equal learning in Deep Q-learning?

In my current research project I'm using the Deep Q-learning algorithm. The setup is as follows: I'm training the model (using Deep Q-learning) on a static dataset made up of experiences extracted ...
user avatar
  • 81
1 vote
3 answers
75 views

Is my model underfitting?

Model: ...
user avatar
  • 23
1 vote
0 answers
24 views

Reducing High generalization-error on industrial fault data

I have a industrial dataset containing labeled machine data for fault classification(3 classes: 1 ok, 2 for faults). The problem is that i have less (~16) different machines, thus iam currently having ...
user avatar
  • 41
8 votes
2 answers
1k views

High accuracy on test-set, what could go wrong?

You are given a pre-trained binary ML classification model with 99% accuracy on the test-set (assume the customer required 95% and that the test-set is balanced). We would like to deploy our model in ...
user avatar
2 votes
1 answer
140 views

Multilabel Classification - Overfitting?

My task is the following: To input drug combinations and output renal failure-related symptoms from the drug combinations. Both the drug combinations and renal-failure related symptoms are represented ...
user avatar
  • 1,311
-1 votes
1 answer
105 views

Why when my local cv of loss decreases, my leaderboard's loss increases?

I got a cv log_loss of 0.3025410331400577 when using 4-fold cross-validation and my leaderboard (with 30% of test dataset) got 0.26514. I further did feature engineering and added some features to the ...
user avatar
1 vote
1 answer
36 views

What is the form of data used for prediction with generalized stacking ensemble?

I am very confused as to how training data is split and on what data level 0 predictions are made when using generalized stacking. This question is similar to mine, but the answer is not sufficiently ...
user avatar
1 vote
1 answer
112 views

Upper bound on size of sample set for decision trees

Say I have an instance space with 4 features and I know that a decision tree with 8 nodes can represent the target function I want to learn. I want to give an upper bound on the size of the sample set ...
user avatar
  • 121
0 votes
1 answer
730 views

how to shuffle the data for model.fit with custom data generator?

So trainfiles is a list that contains the files' directory and name e.g. ['../train/1.npy' , '../train/2.npy'] and then I create a dataset as shown in the middle of the code then I apply it to model ...
user avatar
0 votes
2 answers
513 views

Compare Classification Performance in Datasets drawn from Different Populations

I've read some classics about comparison of ML Algorithms i.e. ...
user avatar
  • 69
2 votes
1 answer
183 views

Why does this paper say that 0-1 loss is insensitive to scaling of weights in a neural network?

When discussing capacity control using norms of weights in a neural network,this paper says the following(see P4): Capacity control in terms of norm, when using a zero/one loss (i.e. counting ...
user avatar
  • 121
1 vote
2 answers
79 views

Will oversampling help with generalization (small imbalanced dataset)?

I have an imbalanced dataset (2:1 ratio) with about 60 patients and 80 features. I performed Recursive Feature Elimination (RFE) and stratified cross validation to reduce the features to 15 and I get ...
user avatar
0 votes
1 answer
65 views

Are there some research papers about text-to-set generation?

I have googled but find no results. Text-to-(word)set generation or sequence-to-(token)set generation. For example, input a text and then output the tags for this text: ...
user avatar
  • 561
5 votes
1 answer
45 views

On design of the training set: conceptual question

I am curious to know how training data should be constructed so that it scales to examples that are not a part of the training data. For example, the problem that I am facing right now is in the ...
user avatar
  • 521
1 vote
0 answers
18 views

How much can be learned from a dataset?

Is there a way to quantify or characterize some upper limits for how much one can "learn" from a dataset? I got puzzled, when the neural state machine got me thinking components of AGI and lead me ...
user avatar
  • 21
1 vote
1 answer
365 views

Generalization of RNN/LSTM/GRU... model

Given a time-series prediction with a Recurrent Neural Network (doesn't matter if LSTM/GRU/...), a forecast might look like this: to_predict (orange) was fed to the model, predicted (purple) is the ...
user avatar
  • 111
0 votes
1 answer
36 views

Is generalizing a model, then removing the generalization good for FFNNs?

If one is training a basic FFNN (Feed-Forward Neural Network), one would apply regularizations like dropout, l1, l2 and gaussian noise, so that the model is robust and gives better results for unseen ...
user avatar
2 votes
1 answer
257 views

XGBoost: what to do when Kfold is not enough?

I have a dataset made of roughly 100 time-series and my final goal is to obtain a classification of each point (detection problem). To do so I have labels so I decided to use an XGB model to perform ...
user avatar
0 votes
1 answer
204 views

How can I measure the reliability of the specificity of a model with very small train, test, and validation datasets?

Stats newbie here. I have a small dataset of 646 samples that I've trained a reasonably performant model on (~99% test and val accuracy). To complicate things a little bit, the classes are somewhat ...
user avatar
  • 312
1 vote
0 answers
63 views

Network either overfits or underfits, but never generalizes - what to do?

I have a simple network with 1st level an LSTM, dropout, fully-connected and softmax layers; loss is cross-entropy (four classes, well balanced). Sequence length to LSTM is 172 samples, data is z-...
user avatar
3 votes
1 answer
536 views

What is NLP technique to generalize manually created rules in text?

Let's say we have a free text containing key-value entities. Example: "... patient's tumour has width 6 cm and height 5 cm" Then an expert comes, marks it as important, thus we do have the rule for ...
user avatar
  • 133
5 votes
1 answer
1k views

Training on accurate data versus noisy data

I have data currently available that is very accurate and I would like to train my classification methods on this set of clean data to learn the important markers for distinguishing between classes. ...
user avatar
  • 185
0 votes
2 answers
194 views

Is a Neural Network with 20 times the number of input neurons (on hidden layers) guaranteed to overfit? When is this not so?

I'm aware of the problem of over-fitting. One way to describe it is your Neural Network learning your training data to a high accuracy and performing poorly (generalizing) on new data. Was wondering ...
user avatar
0 votes
2 answers
533 views

Why is it bad to use the same test dataset over and over again?

I am following this Google's series: Machine Learning Crash Course. On the chapter about generalisation, they make the following statement: Good performance on the test set is a useful indicator of ...
user avatar
1 vote
2 answers
265 views

How is the equation for the relation between prediction error, bias, and variance defined?

I'm reading this article Understanding the BiasVariance Tradeoff. It mentioned: If we denote the variable we are trying to predict as $Y$ and our covariates as $X$, we may assume that there is a ...
user avatar
12 votes
2 answers
3k views

The differences between SVM and Logistic Regression

I am reading about SVM and I've faced to the point that non-kernelized SVMs are nothing more than linear separators. Therefore, ...
user avatar
  • 5,584
5 votes
1 answer
1k views

The connection between optimization and generalization

Optimization algorithms such as gradient descent or particle swarm can find a minima in a function. On the other hand, learning methods such as back-prop define learning as an optimization problem ...
user avatar