Questions tagged [model-selection]

Model selection is the process of comparing several models and their respective results to choose the model is best according to some evaluation metric.

Filter by
Sorted by
Tagged with
2 votes
1 answer
17 views

Nested Cross-Validation with Small dataset

I am currently working with a small dataset (only 175 samples, 45 features) and have been reading on the proper way to cross-validate my model. I had started with a basic cross-validation using a grid ...
user avatar
0 votes
0 answers
6 views

additive or the multiplicative component model

My task: I have to decide that the additive or the multiplicative component model (with a nonparametric trend function but without seasonality) is more suitable for my time series. I found these ...
user avatar
0 votes
0 answers
6 views

I need more than one model to help me understand the relationship and the effect between the variables [closed]

I intend to study the impact in the short and long term of three dependent variables (overall financial development index , financial market development variable, and institutional development index) ...
user avatar
0 votes
0 answers
10 views

Choosing a model to forecast parallel time series with multiple features

I have 6 websites, and I am trying to forecast the number of chat bots opened per hour for each website. The time forecast is 72 hours later. Data Format There are 15,000 data points (deseasonalised),...
user avatar
0 votes
0 answers
15 views

Changing a 0-1 column datatype from int64 to uint8 such as in pandas.get_dummies()

Is it advisable to change the datatype int64 of a 0-1's column to uint8 such as ...
user avatar
0 votes
0 answers
16 views

Why do i need to write regressor.predict(x_train)?

Im currently learning data science and i was unable to understand a particular part in linear regression model. The following is my code - ...
user avatar
  • 1
2 votes
0 answers
18 views

Capping labels negatively impacts business metric

I have this deep neural network model with an integer label to predict. The label is heavily skewed so we cap the labels at some value (let's say 90 %ile). Now when we build and run the model, it ...
user avatar
  • 131
0 votes
1 answer
40 views

How can I export the best classifier from my code to a model for real future usage?

...
user avatar
  • 1
0 votes
0 answers
15 views

Neural Network for solving these linear algebra problems

Intro There are several questions on this site about whether or not machine learning can solve specific problems. The answer (in my words) seems to be: "Yes, trivially, if you choose a model to ...
user avatar
0 votes
2 answers
19 views

do feature selection and model selection must share the same ratio between development set and test set?

As the title, after I performed a Feature Selection, is it mandatory to respect the same ratio (between development set and test set) in Model Selection?
user avatar
0 votes
0 answers
20 views

Model works on TF 2.3 but not on 2.6 ( model.predict_classes removed?)

I am writing a project that classifies the date codes on a pack, I have developed a pipeline that works as intended on my PC, I trained the model on my computer and ran the classification script (tf2....
user avatar
  • 1
1 vote
1 answer
15 views

How to choose Recursive Feature Elimination parameters

in my project I have >900 features and I thought to use Recursive Feature Elimination algorithm to reduce the dimensionality of my problem (in order to improve the accuracy). But I can't figure out ...
user avatar
0 votes
0 answers
9 views

How to check the validity of data? if There are repeated Y values against X data

I need to find the best-fitted curve for this data, I am not sure which model should I use? Can someone suggest me a model? I also have doubts that this dataset is not valid. As there are multiple Y ...
user avatar
  • 101
0 votes
1 answer
18 views

Models: during training and during deployment

It's known that during the model training, we hold out the test-set. However, I actually find during deployment, that if to use a new model train on the entire dataset (train+test), actually yield ...
user avatar
  • 359
1 vote
1 answer
26 views

Feature selection with "overly important" features

I am very new to machine learning modeling, but I encountered a feature selection problem that I hope can get your insights on: For example, I have A,B,C,D as my independent variables and y as my ...
user avatar
2 votes
1 answer
78 views

Is data leakage giving me misleading results? Independent test set says no!

TLDR: I evaluated a classification model using 10-fold CV with data leakage in the training and test folds. The results were great. I then solved the data leakage and the results were garbage. I then ...
user avatar
  • 21
0 votes
0 answers
31 views

Repeatability tests for machine learning models (in the sense of measurement system analysis)

For analyzing a machine learning model, we usually calculate the model performance metrics (such as accuracy...) and during validation step make sure that the model has not overfitted. We can consider ...
user avatar
  • 1
0 votes
0 answers
13 views

How to approach a spatio-temporal forecasting?

I am dealing with a Spatio-temporal forecasting problem similar to the one dealing with the NYC Taxi Demand Prediction. This case is a good example since it has been already covered in different ...
user avatar
0 votes
0 answers
19 views

How to manage sampling bias between training data and real-world data?

I'm currently working on a binary classification problem. My training dataset is rather small with only 1000 elements. (I don't know if it is relevant : my problem is similar to the "spam ...
user avatar
  • 1
0 votes
1 answer
35 views

Remove frame from background

I am having 400 images that look like the following: I would like to remove the frame and only get the image in the middle: I tried the MODNet model ...
user avatar
  • 187
0 votes
0 answers
9 views

How to properly measure forecast errors when predicting correlation coefficient?

My task is to accurately predict correlation coefficient value. I have some candidate models, and want to select the best one (with minimal forecast errors on validation dataset). I don't feel good ...
user avatar
1 vote
0 answers
42 views

What model to train to restore MNIST test dataset

I came across this problem, and not sure where to start. What model would work best for this problem and why? Imagine the digits in the test set of the MNIST dataset (http://yann.lecun.com/exdb/mnist/)...
user avatar
1 vote
0 answers
23 views

How to go about predicting administrative fees?

We collect administrative fees from our customers based on many complex business rules albeit based on few variables. I have the history of fees colected through time (about 500 records for each ...
user avatar
  • 111
4 votes
1 answer
224 views

How can I choose the best machine learning algorithms from all kinds of algorithms?

When I want to find a model for my data set, I find that there are lots of algorithms that I can use. I know how to minimize selection choices by separating supervised and unsupervised algorithms and ...
user avatar
  • 43
0 votes
1 answer
23 views

how to choose the best machine learning algorithms from all kinds of algorithms? [duplicate]

guys, I am a beginner at data science and I’ve been learning machine learning for a while with some courses online without any help of a teacher and after I’ve got to work with some real projects on ...
user avatar
  • 43
0 votes
0 answers
32 views

Is it possible to derive anything useful from this piece of data?

Let's say you have online Profile A. Profile A is present on 3 websites: X, Y, Z. ...
user avatar
  • 101
1 vote
2 answers
29 views

Which regression models should be used with very tiny dataset?

I have a very tiny dataset to make a regression model. only 22 data points with just 2 float features and 1 float output. I want to make models among sklearn ...
user avatar
  • 43
1 vote
0 answers
20 views

EDA and Attribute selection [closed]

I have a dataset regarding traffic-violations. The attributes are as follows: ...
user avatar
3 votes
1 answer
162 views

What are the most known ML-models that use complex numbers? (if there are any)

Basically just the header. The question is out of curiosity as I haven't seen one yet.
user avatar
0 votes
1 answer
11 views

How can I learn to better explain architectural choices?

I've found out that most of the choices made during model selection are based on a sort of trial and error. From what I've heard, even the most experienced Data Scientists cannot know beforehand ...
user avatar
0 votes
0 answers
55 views

Silhouette Score for different Clustering algorithms

I am trying to compare different clustering algorithms on a dataset and compare the model performance. Since the dataset is quite big (56 features), I applied PCA to reduce the number of features to ...
user avatar
  • 45
0 votes
0 answers
22 views

average of two models with training set N/2 vs one model with training set N

I'm new to ML and I got a question about training model. Imagine linear regression $Y=\beta^TX+ \epsilon$ and we have training set D (size=N). I have two options: Train model use whole D and we get $\...
user avatar
  • 1
0 votes
1 answer
18 views

Assess the goodness of a ML generative model (text)

Take a RNN network fed with Shakespeare and generating Shakespeare-like text. Once a model seems mathematically fine, as can be assessed by observing its loss and accuracy over training epochs, how ...
user avatar
  • 143
1 vote
2 answers
49 views

Selecting Training Set from Model CV Error

What I would like to do is recursively: Train the model on all data Remove the sample(s) with highest error Repeat until the remaining samples have an acceptable error The hypothesis is: "To ...
user avatar
0 votes
0 answers
13 views

Creating a metric to compare models based on score and time

I have several models from which I need to choose the 'best' one. I am trying to find a metric that can define mathematically what 'best' means. The two parameters to be considered are ...
user avatar
  • 11
1 vote
2 answers
54 views

Why cant we further tune/change the model after evaluating on the test set?

Every thread on stackexchange that I've found says that you can only use the test set once and thats it. So for instance, if you used a linear regression model and got poor results on the test set, ...
user avatar
  • 13
0 votes
0 answers
9 views

Is it the case that ACF and PACF reflect full information about ARIMA model parameters (p,q)?

Let say i have a single time series of N observations. I'm wondering how informative are ACF and PACF functions of this series. As we know, they can be used to infer orders of AR and MA part of ARIMA ...
user avatar
1 vote
1 answer
14 views

Would all classification models perform similarly in a theoretical and ideal scenario?

Imagine that we count on infinite computation power, an infinite amount of data and we have an infinite amount of time to wait for a model to learn. In such a scenario, we want to have some data ...
user avatar
  • 243
0 votes
0 answers
20 views

What analytics model/classifier should be used to predict price?

I've go dataset with more than 10 features - football skills describing players. There is also price value in each row. I would like to predict price by specifying ...
user avatar
  • 1
1 vote
1 answer
19 views

Measure performance of classification model for training on different snapshots

I am trying to do binary classification on some chronological data. Let's assume we have weekly data from the first week of 2017 through the last week of 2020. Now we have found out that 26 weeks of ...
user avatar
  • 189
0 votes
0 answers
14 views

Which multiclass classification to use for this problem with 9k+ classes?

Need help with which machine learning algorithm/model to use for this problem. The dataset is of product categorization for Amazon. Feature Columns are PRODUCT NAME, PRODUCT DESCRIPTION, BULLET_POINTS,...
user avatar
2 votes
0 answers
25 views

Why are correlation matrices used versus a matrix of R^2 values?

I'm relatively new to DS, so forgive me if this is a dumb question or in the wrong forum When evaluating features it seems that almost everywhere a correlation matrix is used ...
user avatar
  • 21
7 votes
1 answer
102 views

Ethical consequences of non-deterministic learning processes?

Most advanced supervised learning techniques are non-deterministic by construction. The final output of the model usually depends on some random parts of the learning process. (Random weight ...
user avatar
  • 2,145
0 votes
2 answers
203 views

How should I implement machine learning for multi-tenant website?

The company I work for has a website for personal use to track leads and opportunities. I implemented a linear regression algorithm to predict a score for opportunities which is trained on the ...
user avatar
  • 111
0 votes
1 answer
44 views

How to classify ordered labels(ordinal data)?

I have some data similar to movie ratings and the labels are ordered, like 1 to 10. since the target label is not a nominal but ordinal variable, what types of models should I be using for classifying ...
user avatar
0 votes
1 answer
853 views

TypeError: __init__() missing 1 required positional argument: 'num_features'

I was trying to denoise image using Deep Image prior. when I use ResNet as an architecture i am getting error. ...
user avatar
  • 103
0 votes
2 answers
43 views

How can compare suggestion models with different performances?

I have 4 class binary classification models. That models identify which class a particular students is suitable for. For example, we have user 1 and 4 classes ...
user avatar
  • 101
0 votes
1 answer
38 views

What is better opensource alternative for identifying small face other than yolo?

I was trying to identify small face meaning that I want to know who that face belong to according to training dataset. I have previously use yolov4 to detect small object before and I know the ...
user avatar
  • 37
1 vote
0 answers
49 views

Find the relation between two logical models, and their inductive bias

Suppose we want to learn the Boolean function in instance space $X=\{0,1\}^3$. We are given two models to examine: $H_1$ is a set of all logical functions in the conjunctive normal form (CNF), $H_2$ ...
user avatar
1 vote
1 answer
36 views

Applying an algorithm on it's own training data for descriptive purposes?

Good Day, I am newer to data science so I am not confident in this. To set up the question I will describe my data and approach. Data I don't want to share specific data examples as I want to try and ...
user avatar

1
2 3 4 5