Questions tagged [model-selection]

Model selection is the process of comparing several models and their respective results to choose the model is best according to some evaluation metric.

Filter by
Sorted by
Tagged with
1
vote
1answer
48 views

Model for predicting duration based on categorical data

I am working on a model which will allow me to predict how long it will take for a "job" to be completed, based on historical data. Each job has a handful of categorical characteristics (all ...
0
votes
1answer
246 views

Population stability Index vs Population Accuracy Index

Can anyone explain to me the difference between Population Stability Index(PSI) and Population Accuracy Index(PAI)?
0
votes
1answer
26 views

Remove frame from background

I am having 400 images that look like the following: I would like to remove the frame and only get the image in the middle: I tried the MODNet model ...
0
votes
2answers
109 views

How should I implement machine learning for multi-tenant website?

The company I work for has a website for personal use to track leads and opportunities. I implemented a linear regression algorithm to predict a score for opportunities which is trained on the ...
4
votes
1answer
179 views

How can I choose the best machine learning algorithms from all kinds of algorithms?

When I want to find a model for my data set, I find that there are lots of algorithms that I can use. I know how to minimize selection choices by separating supervised and unsupervised algorithms and ...
2
votes
1answer
601 views

Optimizing decision threshold on model with oversampled/imbalanced data

I'm working on developing a model with a highly imbalanced dataset (0.7% Minority class). To remedy the imbalance, I was going to oversample using algorithms from imbalanced-learn library. I had a ...
1
vote
2answers
65 views

How do I select the "best" unsupervised machine learning algorithm to cluster my specific dataset?

I want to cluster a dataset without prior knowledge on the correct amount of clusters. For different algorithms (i.e. k-means, gmm...) I can iterate through different values and try to find the best ...
1
vote
3answers
228 views

Retrieve user features in real time from UserId for prediction

Let's say I'm building an app like Uber and I want to predict the user's most likely destination based on the user's past history, current latitude/longitude, and time/date. Here is the proposed ...
1
vote
2answers
294 views

Same validation accuracy, different train accuracy for two neural networks models

I'm performing emotion classification over FER2013 dataset. I'm trying to measure different models performance, and when I checked ImageDataGenerator with a model I had already used I came up with the ...
1
vote
1answer
35 views

What probability distribution would be more appropriate for monthly rate of going to the store?

Part of a model I am making includes the frequency with which people go shopping for a given good (e.g. people on average go to the supermarket some n times a month). I am trying to figure out what ...
0
votes
0answers
6 views

How to properly measure forecast errors when predicting correlation coefficient?

My task is to accurately predict correlation coefficient value. I have some candidate models, and want to select the best one (with minimal forecast errors on validation dataset). I don't feel good ...
4
votes
1answer
115 views

Neural Network - Sparsity of collaborative based filtering and modelling the prediction problem

I'm fairly new to machine learning and for that matter, neural networks, but for the past couple of days I decided to take a stab at a fairly classical and practical problem of neural networks/machine ...
1
vote
0answers
23 views

What model to train to restore MNIST test dataset

I came across this problem, and not sure where to start. What model would work best for this problem and why? Imagine the digits in the test set of the MNIST dataset (http://yann.lecun.com/exdb/mnist/)...
1
vote
1answer
166 views

Regression model for continuous dependent variable and count independent variables

I am currently learning R and I am relatively inexperienced in the field. Hope I can get some advice from you guys! I am working on a project where I have to estimate the average processing time of ...
1
vote
1answer
85 views

Probabilistic Machine Learning model to match spatial data

I have spatial data from multiple sources. This data consists of ID, lat, long, and time. My goal is that given a new lat-long, the model needs to return (preferably with a probability) the data ...
3
votes
4answers
206 views

ML: Classification Model Comparison

Given is a dataset that I need to use for a classification and I want to compare the performance of different classification models. Let's assume, I want to look at logistic regression (with ...
6
votes
6answers
1k views

Is there any way to explicitly measure the complexity of a Machine Learning Model in Python

I'm interested in model debugging and one of the points that it mentions is to compare your model with a "less complex" one in order to check if the performance is substantially better on ...
1
vote
1answer
102 views

Is it a good practice to evaluate model performance by comparing the metrics of rescaled (inverse transformed) predictions and true target values?

I am now working with a Linear Regression for a time-series regression problem (I am sorry that I cannot say too much about the problem and feature vector due to NDA). I scaled both the input values ...
1
vote
1answer
49 views

Propensity model with Only Positive Data

Is it possible to build a propensity model (i.e., the likelihood that a user will buy an item) using only positive values. For example, I have a bunch of data about Customers (people that bought stuff)...
0
votes
1answer
78 views

What model is recommended: I am using text features in a regression and want to interpret coefficients

I am using the text of comments on a forum to predict how many upvotes it will get. I want to be able to say, "Reviews with X, Y, Z words are more upvoted". So to do this, I want to use text features ...
1
vote
0answers
22 views

How to go about predicting administrative fees?

We collect administrative fees from our customers based on many complex business rules albeit based on few variables. I have the history of fees colected through time (about 500 records for each ...
0
votes
1answer
20 views

how to choose the best machine learning algorithms from all kinds of algorithms? [duplicate]

guys, I am a beginner at data science and I’ve been learning machine learning for a while with some courses online without any help of a teacher and after I’ve got to work with some real projects on ...
3
votes
1answer
178 views

Transfer Learning Question: Extending the Functionality of a Multipose-Estimation Machine Learning Model?

I have experimented with a number of different machine learning models used for pose estimation. Most of them output a heatmap and offsets for the detected person(s) in the image. I really like the ...
0
votes
0answers
31 views
2
votes
1answer
72 views

Ranking ATM based on Utilization and Economic Data (Scoring/Rank Model)

I have a sample data of around 10 ATM's Locations along with their Utilization Count (Deposits, Withdrawals and Others) for the past 3 months. I am planning to collect additional data such as nearby ...
3
votes
2answers
483 views

Where does the "deep learning needs big data" rule come from

When reading about deep learning I often come across the rule that deep learning is only effective when you have large amounts of data at your disposal. These statements are generally accompanied by a ...
3
votes
2answers
82 views

How can I approach this problem?

Let's say I have a dataset with pricing information for the same flight during the past year. So, for a flight departing on day D, I have the available price from D-130 up to D (departure day). Then ...
1
vote
1answer
21 views

Which regression models should be used with very tiny dataset?

I have a very tiny dataset to make a regression model. only 22 data points with just 2 float features and 1 float output. I want to make models among sklearn ...
0
votes
1answer
65 views

I have hourly data of a metric for 15 days, Can i predict the outcome values for same metric for the next 15 days?

I have tried a linear regression model for the same data, Since the regression line is continuous i'm not sure if it works to predict the outcome values for next 15 days, or for a given period of time!...
1
vote
0answers
16 views

EDA and Attribute selection [closed]

I have a dataset regarding traffic-violations. The attributes are as follows: ...
3
votes
3answers
91 views

How best to show the best model over multiple labels?

I have 4 models I trained and I want to display their prediction success over 45 different labels I tested them on. I get a very messy plot when I naively try to place them one on top of the other. ...
3
votes
2answers
1k views

Is k-means with Mahalanobis a valid option for clustering?

I want more info into if k-means with Mahalanobis distance is a mathematically/methodologically correct option for datasets with different variance clusters. The steps are: Create aggregate datasets (...
3
votes
2answers
214 views

Model selection and assessment using leave-one-out cross validation

Assume $D$ is the training data set with both the value of the predictors $\mathbf{X}$ and the value of the response variable $Y$. I have a loss function $L$ and two models $f(\mathbf{X};\beta)$ and $...
3
votes
1answer
79 views

What are the most known ML-models that use complex numbers? (if there are any)

Basically just the header. The question is out of curiosity as I haven't seen one yet.
0
votes
1answer
11 views

How can I learn to better explain architectural choices?

I've found out that most of the choices made during model selection are based on a sort of trial and error. From what I've heard, even the most experienced Data Scientists cannot know beforehand ...
1
vote
1answer
550 views

checking model stability - Performance for different class

I tried to do multi-class classification problem. The goal is to predict whether the match will be won by HomeTeam, AwayTeam or Draw. I did feature engineering from the attributes and finally came up ...
1
vote
1answer
46 views

Model Selection using Bias Variance Trade Off

I have a Regression Model with Train MAPE as 6% and Test MAPE as 15%. This appears to me as a clear case of over fitting. But can I still use this model assuming 15% Error is not a bad number after-...
1
vote
1answer
93 views

Finding the equation for a multiple and nonlinear regression model?

Regarding nonlinear and multivariable regression, I am using R or Matlab. In the case where I have a regression with just two variables, I simply draw the graph Y with respect to X and look for the ...
0
votes
0answers
36 views

Silhouette Score for different Clustering algorithms

I am trying to compare different clustering algorithms on a dataset and compare the model performance. Since the dataset is quite big (56 features), I applied PCA to reduce the number of features to ...
1
vote
2answers
49 views

Selecting Training Set from Model CV Error

What I would like to do is recursively: Train the model on all data Remove the sample(s) with highest error Repeat until the remaining samples have an acceptable error The hypothesis is: "To ...
0
votes
0answers
20 views

average of two models with training set N/2 vs one model with training set N

I'm new to ML and I got a question about training model. Imagine linear regression $Y=\beta^TX+ \epsilon$ and we have training set D (size=N). I have two options: Train model use whole D and we get $\...
0
votes
1answer
210 views

How to compare performance between SVM and Keras models

I applied both SVM and CNN (using Keras) on a dataset. Now, I want to compare the performance of both models. Keras model.evaluate function predicts the output for the given input and then computes ...
0
votes
1answer
13 views

Assess the goodness of a ML generative model (text)

Take a RNN network fed with Shakespeare and generating Shakespeare-like text. Once a model seems mathematically fine, as can be assessed by observing its loss and accuracy over training epochs, how ...
0
votes
0answers
12 views

Creating a metric to compare models based on score and time

I have several models from which I need to choose the 'best' one. I am trying to find a metric that can define mathematically what 'best' means. The two parameters to be considered are ...
1
vote
2answers
45 views

Why cant we further tune/change the model after evaluating on the test set?

Every thread on stackexchange that I've found says that you can only use the test set once and thats it. So for instance, if you used a linear regression model and got poor results on the test set, ...
1
vote
0answers
57 views

Best model for Antimicrobial Resistance (AMR) prediction?

Some classes of problem are best solved by a specific class of machine learning model, due to the structure of the data (e.g. CNN's for computer vision tasks). Prediction of bacterial resistance/...
0
votes
0answers
9 views

Is it the case that ACF and PACF reflect full information about ARIMA model parameters (p,q)?

Let say i have a single time series of N observations. I'm wondering how informative are ACF and PACF functions of this series. As we know, they can be used to infer orders of AR and MA part of ARIMA ...
1
vote
1answer
14 views

Would all classification models perform similarly in a theoretical and ideal scenario?

Imagine that we count on infinite computation power, an infinite amount of data and we have an infinite amount of time to wait for a model to learn. In such a scenario, we want to have some data ...
5
votes
0answers
68 views

Ethical consequences of non-deterministic learning processes?

Most advanced supervised learning techniques are non-deterministic by construction. The final output of the model usually depends on some random parts of the learning process. (Random weight ...
1
vote
1answer
13 views

Measure performance of classification model for training on different snapshots

I am trying to do binary classification on some chronological data. Let's assume we have weekly data from the first week of 2017 through the last week of 2020. Now we have found out that 26 weeks of ...

1
2 3 4 5