Questions tagged [model-selection]
Model selection is the process of comparing several models and their respective results to choose the model is best according to some evaluation metric.
257
questions
0
votes
0
answers
31
views
Existence of a "three-point" machine learning model?
I may want to ask if there are studies that exist which utilize a "three-point machine learning model. What I mean by "three-point machine learning model is that it may use several ...
2
votes
2
answers
90
views
Group unstructured chat logs into conversations
I am new to ML/AI/NLP and am interested in tackling the following problem. I have a database of chat logs from a Discord server. The database contains the following labeled data: ...
- 21
0
votes
0
answers
14
views
Which CNN model to use for the classification(20 classes) of gemstones (diamonds, sapphire, ruby etc) based on digital photo images and huge data set?
Im trying to build CNN Model for the classification of precious stones (like diamonds, sapphire, ruby) based on digital images. So I have data set of labeled 150,000 gemstone certifications and the ...
- 1
0
votes
0
answers
48
views
How to make the model fitting to the inverse-square law in Python?
I am studying basics of physics. I would like to learn to make a graphical solution to the following problem:
In a class room a lighting meter was used to measure the illuminance E at the distance r. ...
0
votes
1
answer
37
views
Do I need to use always the same "Test" dataset to compare between different models?
I have two datasources A and B, and I want to check how several methods can affect the accuracy of my multi class models:
If I use cross-validation with validate dataset to obtain the best
hyper ...
0
votes
0
answers
20
views
LSTM input_shape returns value error
My time series dataset dimension are as follows:
print(X_train.shape) = (1766, 4) i.e. 1,766 time steps and 4 features
...
0
votes
1
answer
43
views
How does GridSearchCV use Cross Validation to produce a Model's Score?
I understand Cross Validation in practice, but I'm not sure how SciKit-Learn's GridSearchCV uses it to produce an overall score/ metric for a model. For example, if ...
- 523
1
vote
1
answer
45
views
Good flowchart for which ML model to use given characteristics of the dataset?
There are so many ML models to choose from. Looking for a flow chart that someone may have created or come across that helps you decide which ML model(s) to use.
Here is some of the possible flow ...
- 111
0
votes
1
answer
34
views
Models that are good for long answer generation given context and question and what datasets would be the best for training?
Basically I am trying to create a context-needing question and long answer model and I was wondering what model would be best for such tasks, currently I am leaning towards T5, or GPT-NeoX-20B. ...
0
votes
0
answers
15
views
Image recognition model with CNN for face gestures is really bad
I have a dataset that contains facial expressions and their label, and I am trying to make a classification model for it. Unfortunatly, I can't manage to create a good model with CNN, as the highest ...
0
votes
0
answers
3
views
Model plus client's prior recomendation
The following question is from real-life experience dealing with clients to create a certain model. So say for example I have a data set. The data set has features columns $A,B,C,D$ and labels $L$. ...
- 115
1
vote
1
answer
43
views
Can applying different models and seeing which one fits best be called brute forcing?
I have seen a tutorial which said that you have to try different models and see which fits best on your data.
Can this be considered brute force? I have searched this on google and the closest answer ...
0
votes
0
answers
25
views
Machine learning with mixed variables dataset (numerical, categorical and embeddings)
I'm working on a machine learning project where I'm trying to predict the revenue of a movie.
My dataset contains mixed data types. There are numerical features (rating, number of votes, release year,....
- 101
1
vote
2
answers
94
views
Which is the final model from Nested Cross Validation: Accuracy or Frequency?
https://www.cnblogs.com/guo-xiang/p/8044624.html explains with a nice example the mechanics of Nested Cross Validation.
In the picture, the example shows how to use Nested CV for hyperparameter ...
- 491
0
votes
0
answers
9
views
Choice of proper machine learning model for signal processing applications
I have a data set that will be consist of 1D data that comes from the Fourier Transform (FFT) of time domain samples (x-axis is frequency, y-axis is magnitude). In order to classify $N$ different ...
0
votes
0
answers
13
views
How to best include multiple data points from the same customer when predicting churn rate?
I'm trying to make a machine learning model for predicting churn rate on bank data. I don't have credit details available, but I have the following:
Personal details of customer, including updates ...
- 1
0
votes
0
answers
9
views
How many models can be evaluated against a validation set without having a too small confidence interval?
if we have a small and a large validation set with the large has 100 times many samples. Approximately how many models can be evaluated on the large set without having a too small confidence ...
0
votes
0
answers
21
views
Am I calculating LogAUC / pAUC correctly?
Hope you are well.
I was wondering how one would calculate logAUC? I have an implementation but I don't think it's correct.
I'm trying to recreate the metric in this manuscript. See figure 2.
Any help ...
0
votes
0
answers
21
views
Predicting number of women who will give birth on a given day?
I have a set of data with information about women and their expected delivery dates for childbirth.
I have more columns in the table but for simplicity let's just focus on the below and assume that my ...
- 1
0
votes
0
answers
8
views
Digital attribution modeling to understand ecommerce website usage
Ecommerce websites like Walmart, Amazon, etc. have various features for shoppers like "Add to favorites", "Add to wishlist", "Add to cart", "Buy it now", etc. I ...
- 101
0
votes
1
answer
30
views
Good models for predicting whether a customer would make a purchase given details like age, gender, ethnicity, salary, etc?
I have around 30,000 data points and for those data points I have some numerical fields like customer_age, ...
- 1
2
votes
0
answers
34
views
High overfitting, but test metrics are higher
Ml models must strike a balance between predictive power and generalization power. Therefore, I split the data into train/test and calculate metrics on both. Often I see instructions in someone else's ...
- 376
0
votes
1
answer
26
views
How to find significance for Gini coefficient changes?
I'm using the Gini coefficient to evaluate the performance of a model.
Making some changes (feature selection, hyperparameter tuning, etc.) I created variant models with different Gini coefficients.
...
- 594
0
votes
1
answer
21
views
Algorithm suggestion for correlated models
I'm looking for suggestions on how to proceed with predicting on separate but correlated models.
The example I will use is housing data. I have three inputs:
Latitude
Longitude
1-Google Street View ...
- 493
0
votes
1
answer
32
views
Kernel ridge regression (KRR), accuracy scale?
What does a good range for the accuracy score look like for the KRR model?
For example, RMSE produces a value between 0 and 1, where values closer to 0 represent better fitting models. What's the ...
- 101
1
vote
0
answers
18
views
What is the best way for me to classify this audio data?
I have a set of audio data. I would like to classify each audio file based on a half-second of data from a give time period. The audio data is given as counts as a function of time $s(t)$. Right now ...
- 11
0
votes
0
answers
8
views
image classifcation model's depth and width
I wonder how deep and wide deep learning model should be. Where can I possess some information/rules how many layers and how wide they ought to be?
I created basic image classification model with ...
- 1
0
votes
1
answer
80
views
additive or multiplicative model?
let's say if I have two scores $x_1^i$ and $x_2^i$ for each data point $i$, and I need to make a final score/loss function out of it.
Should I use a weighted sum $w_1 x_1^i + w_2 x_2^i$, or their ...
- 165
0
votes
0
answers
53
views
What are some "best practices" for discovering true patterns in data without relying on "scores" as measures of accuracy?
What are some "best practices" for discovering true patterns in data without knowing about them and without relying on "scores" as measures of accuracy?
In university I was always ...
- 416
0
votes
4
answers
49
views
Where to learn which ML task is most appropriate for a problem?
There is now tons of material available on how to do certain (most popular) ML tasks and what kind of output you can expect.
However I found that resources on how to select appropriate ML task/...
- 103
0
votes
0
answers
10
views
Parameter optimization on unsupervised learning with pseudo golden truth variable: What is the reasonably approach for selecting parameters?
I'm using an unsupervised model that was tested on statistics like Gensim's Word2Vec word analogies test:
...
- 31
0
votes
0
answers
10
views
Clustering or classification to identify characteristics that unite first time participants vs repeating participants
I have a dataset with customers that have participated in a promotion campaign with a company for the first time and those who have been participating more than once. They all have the same variables ...
- 101
0
votes
0
answers
13
views
Small dataset: minimum bounding sphere?
I am trying to apply data-science to a biological problem.
The task is made complex by several factors
the dataset is quite small: 173 points.
only 6 of those are True Positive ( although there might ...
- 101
0
votes
0
answers
16
views
How to validate Probabilisitc models when you only need the rankings of the output (Agent - Customer Pairing)?
A given dataset has call details, Agent specific and Customer specific features. The target output is Sale (1) or not (0). The data is highly imbalanced. Our goal is not to maximize number of sales ...
2
votes
1
answer
129
views
Nested Cross-Validation with Small dataset
I am currently working with a small dataset (only 175 samples, 45 features) and have been reading on the proper way to cross-validate my model. I had started with a basic cross-validation using a grid ...
- 21
0
votes
0
answers
41
views
additive or the multiplicative component model
My task: I have to decide that the additive or the multiplicative component model (with a nonparametric trend function but without seasonality) is more suitable for my time series.
I found these ...
0
votes
0
answers
42
views
Choosing a model to forecast parallel time series with multiple features
I have 6 websites, and I am trying to forecast the number of chat bots opened per hour for each website. The time forecast is 72 hours later.
Data Format
There are 15,000 data points (deseasonalised),...
0
votes
0
answers
126
views
Changing a 0-1 column datatype from int64 to uint8 such as in pandas.get_dummies()
Is it advisable to change the datatype int64 of a 0-1's column to uint8 such as ...
0
votes
0
answers
179
views
Why do i need to write regressor.predict(x_train)?
Im currently learning data science and i was unable to understand a particular part in linear regression model.
The following is my code -
...
- 1
2
votes
0
answers
20
views
Capping labels negatively impacts business metric
I have this deep neural network model with an integer label to predict. The label is heavily skewed so we cap the labels at some value (let's say 90 %ile).
Now when we build and run the model, it ...
- 131
0
votes
1
answer
128
views
0
votes
0
answers
26
views
Neural Network for solving these linear algebra problems
Intro
There are several questions on this site about whether or not machine learning can solve specific problems. The answer (in my words) seems to be:
"Yes, trivially, if you choose a model to ...
0
votes
2
answers
24
views
do feature selection and model selection must share the same ratio between development set and test set?
As the title, after I performed a Feature Selection, is it mandatory to respect the same ratio (between development set and test set) in Model Selection?
0
votes
0
answers
137
views
Model works on TF 2.3 but not on 2.6 ( model.predict_classes removed?)
I am writing a project that classifies the date codes on a pack, I have developed a pipeline that works as intended on my PC, I trained the model on my computer and ran the classification script (tf2....
- 1
1
vote
1
answer
17
views
How to choose Recursive Feature Elimination parameters
in my project I have >900 features and I thought to use Recursive Feature Elimination algorithm to reduce the dimensionality of my problem (in order to improve the accuracy).
But I can't figure out ...
0
votes
0
answers
9
views
How to check the validity of data? if There are repeated Y values against X data
I need to find the best-fitted curve for this data, I am not sure which model should I use? Can someone suggest me a model?
I also have doubts that this dataset is not valid. As there are multiple Y ...
- 101
0
votes
1
answer
48
views
Models: during training and during deployment
It's known that during the model training, we hold out the test-set. However, I actually find during deployment, that if to use a new model train on the entire dataset (train+test), actually yield ...
- 397
1
vote
1
answer
29
views
Feature selection with "overly important" features
I am very new to machine learning modeling, but I encountered a feature selection problem that I hope can get your insights on:
For example, I have A,B,C,D as my independent variables and y as my
...
- 13
2
votes
0
answers
108
views
Is data leakage giving me misleading results? Independent test set says no!
TLDR:
I evaluated a classification model using 10-fold CV with data leakage in the training and test folds. The results were great. I then solved the data leakage and the results were garbage. I then ...
- 21
0
votes
0
answers
80
views
Repeatability tests for machine learning models (in the sense of measurement system analysis)
For analyzing a machine learning model, we usually calculate the model performance metrics (such as accuracy...) and during validation step make sure that the model has not overfitted.
We can consider ...
- 1