Questions tagged [generalization]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
1 answer
22 views

Is it possible to train a Support Vector Machine to a specific accuracy?

From my understanding, support vector machines run on the premise of minimizing some error function, usually with the goal of maximizing accuracy overall. However, there are a lot of contexts, ...
  • 27
0 votes
1 answer
31 views

Why an already trained model is not generalizable to another related dataset?

A model is trained to predict the median temperature of Boston. The resulting model works well according to their validation data. However, this model performs poorly when used to predict the ...
  • 1,267
0 votes
1 answer
48 views

Why does the application of the logarithmic function improve the outcome of Random forests?

I have a Random forest model that tries to predict what kind of a useful activity a machine is doing based on its power readings. There are 5 features in a single reading. There are two main types of ...
  • 21
0 votes
0 answers
22 views

What if many different models reach the same maximum metrics

I am talking about trying different algorithms, different parameters, different stacking configurations that all improve upon previous baselines, and yet a lot of them have exactly the same values on ...
  • 518
1 vote
1 answer
31 views

How does batch normalization make a model less sensitive to hyperparameter tuning?

Question 22 of 100+ Data Science Interview Questions and Answers for 2022 asks What is the benefit of batch normalization? The first bullet of the answers to this is The model is less sensitive to ...
  • 172
0 votes
0 answers
101 views

LGBM model predicting only single class on unseen data!

I have built a LightGBM based machine learning model on data of molecules of two classes. The distribution is as follows. Class 0 has 5933 data points and class 1 has 4696. The train test accuracy I ...
  • 47
0 votes
1 answer
30 views

About neural network ability to generalize

I had this question during an interview that I wasn't able to answer, even after researching on the internet. Which of the following can affect an artificial neural network’s ability to generalize??? ...
2 votes
1 answer
223 views

Which type of models generalize better, generative or discriminative models?

In NLP, which type of models (generative or discriminative) is more sensitive to the amount of data to generalize better? references? This is related to the way those two types capture the data ...
  • 252
0 votes
0 answers
23 views

Can there be scenarios where an overfitted model in machine learning cannot be generalized?

Is it always possible to generalize an overfitted model? I know there are ways to handle overfitting, but can there be scenarios where overfitting cannot be handled in machine learning?
1 vote
2 answers
332 views

Graph Neural Network fails at generalizing on unseen graph topologies

I'm using PytorchGeometric to train a graph convolutional network for regression over nodes problem (the graph models physical phenomena in the network of sensors; the network of sensors is actually ...
  • 11
1 vote
0 answers
13 views

Generalization error problem on training set

Training data: $\mathcal {T} =\{(2,1),(3,2),(4,6),(0,0),(1,1)\}$ you already computed a predictor for the output using linear regression by least squares, where you used the first 3 samples as ...
2 votes
1 answer
211 views

Does convergence equal learning in Deep Q-learning?

In my current research project I'm using the Deep Q-learning algorithm. The setup is as follows: I'm training the model (using Deep Q-learning) on a static dataset made up of experiences extracted ...
  • 81
1 vote
3 answers
82 views

Is my model underfitting?

Model: ...
  • 23
1 vote
0 answers
27 views

Reducing High generalization-error on industrial fault data

I have a industrial dataset containing labeled machine data for fault classification(3 classes: 1 ok, 2 for faults). The problem is that i have less (~16) different machines, thus iam currently having ...
  • 41
9 votes
2 answers
2k views

High accuracy on test-set, what could go wrong?

You are given a pre-trained binary ML classification model with 99% accuracy on the test-set (assume the customer required 95% and that the test-set is balanced). We would like to deploy our model in ...
2 votes
1 answer
229 views

Multilabel Classification - Overfitting?

My task is the following: To input drug combinations and output renal failure-related symptoms from the drug combinations. Both the drug combinations and renal-failure related symptoms are represented ...
  • 1,398
-1 votes
1 answer
149 views

Why when my local cv of loss decreases, my leaderboard's loss increases?

I got a cv log_loss of 0.3025410331400577 when using 4-fold cross-validation and my leaderboard (with 30% of test dataset) got 0.26514. I further did feature engineering and added some features to the ...
1 vote
1 answer
50 views

What is the form of data used for prediction with generalized stacking ensemble?

I am very confused as to how training data is split and on what data level 0 predictions are made when using generalized stacking. This question is similar to mine, but the answer is not sufficiently ...
1 vote
1 answer
133 views

Upper bound on size of sample set for decision trees

Say I have an instance space with 4 features and I know that a decision tree with 8 nodes can represent the target function I want to learn. I want to give an upper bound on the size of the sample set ...
0 votes
1 answer
1k views

how to shuffle the data for model.fit with custom data generator?

So trainfiles is a list that contains the files' directory and name e.g. ['../train/1.npy' , '../train/2.npy'] and then I create a dataset as shown in the middle of the code then I apply it to model ...
0 votes
2 answers
646 views

Compare Classification Performance in Datasets drawn from Different Populations

I've read some classics about comparison of ML Algorithms i.e. ...
  • 69
2 votes
1 answer
286 views

Why does this paper say that 0-1 loss is insensitive to scaling of weights in a neural network?

When discussing capacity control using norms of weights in a neural network,this paper says the following(see P4): Capacity control in terms of norm, when using a zero/one loss (i.e. counting ...
  • 121
1 vote
2 answers
95 views

Will oversampling help with generalization (small imbalanced dataset)?

I have an imbalanced dataset (2:1 ratio) with about 60 patients and 80 features. I performed Recursive Feature Elimination (RFE) and stratified cross validation to reduce the features to 15 and I get ...
0 votes
1 answer
67 views

Are there some research papers about text-to-set generation?

I have googled but find no results. Text-to-(word)set generation or sequence-to-(token)set generation. For example, input a text and then output the tags for this text: ...
  • 631
5 votes
1 answer
45 views

On design of the training set: conceptual question

I am curious to know how training data should be constructed so that it scales to examples that are not a part of the training data. For example, the problem that I am facing right now is in the ...
  • 491
1 vote
0 answers
19 views

How much can be learned from a dataset?

Is there a way to quantify or characterize some upper limits for how much one can "learn" from a dataset? I got puzzled, when the neural state machine got me thinking components of AGI and lead me ...
  • 21
1 vote
1 answer
518 views

Generalization of RNN/LSTM/GRU... model

Given a time-series prediction with a Recurrent Neural Network (doesn't matter if LSTM/GRU/...), a forecast might look like this: to_predict (orange) was fed to the model, predicted (purple) is the ...
  • 111
0 votes
1 answer
38 views

Is generalizing a model, then removing the generalization good for FFNNs?

If one is training a basic FFNN (Feed-Forward Neural Network), one would apply regularizations like dropout, l1, l2 and gaussian noise, so that the model is robust and gives better results for unseen ...
2 votes
1 answer
283 views

XGBoost: what to do when Kfold is not enough?

I have a dataset made of roughly 100 time-series and my final goal is to obtain a classification of each point (detection problem). To do so I have labels so I decided to use an XGB model to perform ...
0 votes
1 answer
208 views

How can I measure the reliability of the specificity of a model with very small train, test, and validation datasets?

Stats newbie here. I have a small dataset of 646 samples that I've trained a reasonably performant model on (~99% test and val accuracy). To complicate things a little bit, the classes are somewhat ...
  • 312
1 vote
0 answers
69 views

Network either overfits or underfits, but never generalizes - what to do?

I have a simple network with 1st level an LSTM, dropout, fully-connected and softmax layers; loss is cross-entropy (four classes, well balanced). Sequence length to LSTM is 172 samples, data is z-...
3 votes
1 answer
609 views

What is NLP technique to generalize manually created rules in text?

Let's say we have a free text containing key-value entities. Example: "... patient's tumour has width 6 cm and height 5 cm" Then an expert comes, marks it as important, thus we do have the rule for ...
  • 133
5 votes
1 answer
1k views

Training on accurate data versus noisy data

I have data currently available that is very accurate and I would like to train my classification methods on this set of clean data to learn the important markers for distinguishing between classes. ...
  • 185
0 votes
2 answers
200 views

Is a Neural Network with 20 times the number of input neurons (on hidden layers) guaranteed to overfit? When is this not so?

I'm aware of the problem of over-fitting. One way to describe it is your Neural Network learning your training data to a high accuracy and performing poorly (generalizing) on new data. Was wondering ...
0 votes
2 answers
767 views

Why is it bad to use the same test dataset over and over again?

I am following this Google's series: Machine Learning Crash Course. On the chapter about generalisation, they make the following statement: Good performance on the test set is a useful indicator of ...
1 vote
2 answers
284 views

How is the equation for the relation between prediction error, bias, and variance defined?

I'm reading this article Understanding the BiasVariance Tradeoff. It mentioned: If we denote the variable we are trying to predict as $Y$ and our covariates as $X$, we may assume that there is a ...
12 votes
2 answers
4k views

The differences between SVM and Logistic Regression

I am reading about SVM and I've faced to the point that non-kernelized SVMs are nothing more than linear separators. Therefore, ...
  • 5,921
5 votes
1 answer
1k views

The connection between optimization and generalization

Optimization algorithms such as gradient descent or particle swarm can find a minima in a function. On the other hand, learning methods such as back-prop define learning as an optimization problem ...