Questions tagged [model-selection]

Model selection is the process of comparing several models and their respective results to choose the model is best according to some evaluation metric.

Filter by
Sorted by
Tagged with
0
votes
0answers
19 views

What analytics model/classifier should be used to predict price?

I've go dataset with more than 10 features - football skills describing players. There is also price value in each row. I would like to predict price by specifying ...
1
vote
1answer
10 views

Measure performance of classification model for training on different snapshots

I am trying to do binary classification on some chronological data. Let's assume we have weekly data from the first week of 2017 through the last week of 2020. Now we have found out that 26 weeks of ...
0
votes
0answers
9 views

Which multiclass classification to use for this problem with 9k+ classes?

Need help with which machine learning algorithm/model to use for this problem. The dataset is of product categorization for Amazon. Feature Columns are PRODUCT NAME, PRODUCT DESCRIPTION, BULLET_POINTS,...
2
votes
0answers
24 views

Why are correlation matrices used versus a matrix of R^2 values?

I'm relatively new to DS, so forgive me if this is a dumb question or in the wrong forum When evaluating features it seems that almost everywhere a correlation matrix is used ...
4
votes
0answers
41 views

Ethical consequences of non-deterministic learning processes?

Most advanced supervised learning techniques are non-deterministic by construction. The final output of the model usually depends on some random parts of the learning process. (Random weight ...
0
votes
1answer
38 views

How should I implement machine learning for multi-tenant website?

The company I work for has a website for personal use to track leads and opportunities. I implemented a linear regression algorithm to predict a score for opportunities which is trained on the ...
0
votes
1answer
23 views

How to classify ordered labels(ordinal data)?

I have some data similar to movie ratings and the labels are ordered, like 1 to 10. since the target label is not a nominal but ordinal variable, what types of models should I be using for classifying ...
0
votes
1answer
56 views

TypeError: __init__() missing 1 required positional argument: 'num_features'

I was trying to denoise image using Deep Image prior. when I use ResNet as an architecture i am getting error. ...
0
votes
2answers
33 views

How can compare suggestion models with different performances?

I have 4 class binary classification models. That models identify which class a particular students is suitable for. For example, we have user 1 and 4 classes ...
0
votes
0answers
114 views

yolo v4 vs yolo v4-tiny which is better for real time object detection or is that any that is better for real time object detection?

When I am comparing Yolo v4 and Yolo v4 tiny, I notice that the tiny one does significantly worse. It may be caused by the small amount of data I use for testing, so I want to ask in a normal amount ...
0
votes
1answer
24 views

What is better opensource alternative for identifying small face other than yolo?

I was trying to identify small face meaning that I want to know who that face belong to according to training dataset. I have previously use yolov4 to detect small object before and I know the ...
1
vote
0answers
47 views

Find the relation between two logical models, and their inductive bias

Suppose we want to learn the Boolean function in instance space $X=\{0,1\}^3$. We are given two models to examine: $H_1$ is a set of all logical functions in the conjunctive normal form (CNF), $H_2$ ...
1
vote
1answer
34 views

Applying an algorithm on it's own training data for descriptive purposes?

Good Day, I am newer to data science so I am not confident in this. To set up the question I will describe my data and approach. Data I don't want to share specific data examples as I want to try and ...
3
votes
1answer
76 views

KFold cross validation ambiguity

I just studied K-Fold cross validation technique for finding model parameters and something seemed to be very confusing. Every tutorial I follow says that for K-Fold validation, the whole dataset will ...
2
votes
0answers
17 views

How to modify a Convolutional Neural Network architecture built for a univariate time series to multivariate time series?

I have built a CNN (in combination with a LSTM cell) that takes 1D time series-like data as an input and performs classification. I am obtaining a good performance, but the complete data has actually ...
1
vote
0answers
54 views

Pretrained model for spectrogram images

I'm working on a sound classification problem. For that, I'm converting the audio signals into spectrogram images and using transfer learning to classify them. Currently I'm using pretrained models ...
2
votes
2answers
367 views

is it possible to decide model without any data?

Today I just faced a very unique demand from my superior. He asked me whether I can make a model first before we gather the data for training because we don't have any data yet. I was utterly ...
1
vote
0answers
20 views

What are Some Good ML/Deep Learning Approaches to This Type of Problem [closed]

Problem Definition: You are given a dataset of $N$ different features. Most of the features are actually calculations of their own time series of set length (i.e A 5-time step weighted moving average)...
1
vote
1answer
34 views

Learning Curves and interpretations

I've trained 4 classifiers on an undersampled dataset. I plotted the learning curve for each classifier and I got the following results : I see that for the Log Reg, both curves seem to converge and ...
2
votes
1answer
70 views

Does RandomForest convergence imply I can solve a problem with a NN too?

I'm trying to perform a regression on a dataset, and I've been testing a few models, mostly for practice. I was able to get good results with a RandomForestRegression model, as you can see in the ...
0
votes
1answer
39 views

Machine Learning Method For Creating Chemical Formula

We work at chemical company . We have nearly 3000 chemical formulas which is composed with chemical raw materials. Our chemical formulas is composed of 20-25 raw materials. As you guess, amount of ...
3
votes
1answer
75 views

At what stage are ROC curves used when building machine learning model?

When developing a machine learning model, at what stage are ROC curve with AUC used? Typically I have three data sets train - ...
1
vote
1answer
42 views

Model Selection using Bias Variance Trade Off

I have a Regression Model with Train MAPE as 6% and Test MAPE as 15%. This appears to me as a clear case of over fitting. But can I still use this model assuming 15% Error is not a bad number after-...
0
votes
0answers
44 views

How do I proceed with model selection?

Only data scientist in an organization and I could really use a sounding board here. In Phase One of a project I deployed four models and served their average as the prediction. I used Random Forest ...
1
vote
1answer
19 views

Is there a quantitative way to determine if a class of algorithms tends produce low bias or low variance models?

I understand that some machine learning models tend to be low bias, whereas others tend to be low variance (source). As an example, a linear regression will tend to have low variance error and high ...
0
votes
0answers
22 views

What sort of models work for unsupervised reinforcement learning, or is deep learning the way?

I'm setting out on an adventure to automate the statuses of the lights around my home. The lights should have different brightness in the range [0, 100] depending on some factors, which I have boiled ...
3
votes
2answers
63 views

How do you, analytically, show you are not using too many features?

One of the managers at my company asked if there is a I could include a metric demonstrates that the my model is not using too many Features/Variables. Is there a metric or best practice that does ...
1
vote
0answers
25 views

How to balance time/effort with transformations, feature selection, and models efficacy in nlp? [closed]

Edit: Question has been edited for reopening (see comment section for justification) Being to new text analytics, I haven't gotten the hang of navigating a typical workflow given the longer times ...
0
votes
0answers
48 views

KL Divergence between Predictions and Ground truth

I've got four (non-linear, tree-based) models in production and using the average of them as the served prediction. We get ground truth data immediately. During training the optimized candidate models ...
0
votes
1answer
85 views

Why does a simpler model performs better than a complicated one?

This has happened to me, a complicated model couldn't solve the problem when a simpler one solved it in a few epochs. How is that? I believed that a more complicated model means more number of ...
1
vote
1answer
21 views

Model selection in active learning

I am dabbling in active learning and was wondering how to combine this in seeking out the best architecture for the network. In my understanding, active learning uses a heuristic for selecting the ...
1
vote
0answers
23 views

Comparing different machine learning methods over multiple test datasets with different number of samples [closed]

Say, I have an image dataset (for example, imagenet) and I am training two image recognition models on it. I train a resnet with 10 layers 3 times on it (each time with different random weight ...
0
votes
1answer
93 views

model selection in clustering

I am working on a mall customer segmentation dataset (5 features, 200 rows) using clustering. This dataset does not have any ground truth labels. I had a few doubts regarding clustering: Can I use ...
4
votes
3answers
315 views

Is autocorrelation of residuals a problem in machine learning?

Let's assume I have a random forest model and the residuals of the model are autocorrelated. Is this a problem? As an example, let's assume I have two different random forest models, A and B, with a ...
0
votes
2answers
24 views

Selecting Transforms with sklearn pipelines

So I am currently working on a Data set, and I want to use Pipelines to select the transforms. Here is an example of what I want to do : ...
1
vote
1answer
66 views

Is it a good practice to evaluate model performance by comparing the metrics of rescaled (inverse transformed) predictions and true target values?

I am now working with a Linear Regression for a time-series regression problem (I am sorry that I cannot say too much about the problem and feature vector due to NDA). I scaled both the input values ...
0
votes
3answers
639 views

Is it a good practice to evaluate a model on the training set

Is it a good practice to evaluate a model on the training set (i.e. train a model on training set and evaluate the regression error/accuracy on the same training set) and compare the evaluation result ...
6
votes
6answers
904 views

Is there any way to explicitly measure the complexity of a Machine Learning Model in Python

I'm interested in model debuggin and one of the points that it recommends is to compare your model with a "less complex" one in order to see if the performance is substantially better on the ...
0
votes
0answers
20 views

Model selection Tensorflow for custom dataset comparing panoramic images vs regular images (Image segmentation)

I have a question regarding the use of a specific model such as Deeplab and how to create a custom dataset for it. Background To give a bit of background info, I want to compare panoramically stitched ...
0
votes
1answer
22 views

Determining which categorical data is beneficial in predictive modelling

I am working on a model which will allow me to predict how long it will take for a "job" to be completed, based on historical data. Each job has a handful of categorical characteristics (all ...
0
votes
1answer
34 views

Model for predicting duration based on categorical data

I am working on a model which will allow me to predict how long it will take for a "job" to be completed, based on historical data. Each job has a handful of categorical characteristics (all ...
0
votes
1answer
170 views

Population stability Index vs Population Accuracy Index

Can anyone explain to me the difference between Population Stability Index(PSI) and Population Accuracy Index(PAI)?
0
votes
1answer
44 views

How do I know what the best number of layers is required to achieve the highest accuracy

I'm learning from Udacity using this video. I saw this piece of code: ...
0
votes
3answers
98 views

how to use predictions on a single value?

I am comfortable using Machine learning on my train data and test data and validate it. But the question here is if I want to predict a single variable how do I do it? Let's suppose I have done ...
1
vote
1answer
27 views

What probability distribution would be more appropriate for monthly rate of going to the store?

Part of a model I am making includes the frequency with which people go shopping for a given good (e.g. people on average go to the supermarket some n times a month). I am trying to figure out what ...
1
vote
1answer
44 views

is it possible get a overfit underfit comparation between models, with this chart? (homework) [closed]

I am trying to interpret this chart. I am not sure how to interpret this, because, I think that the fact of the for examples LGBM Validation error, is wide and similar to train boxplot, there arent ...
1
vote
2answers
48 views

How do I select the “best” unsupervised machine learning algorithm to cluster my specific dataset?

I want to cluster a dataset without prior knowledge on the correct amount of clusters. For different algorithms (i.e. k-means, gmm...) I can iterate through different values and try to find the best ...
1
vote
1answer
37 views

Propensity model with Only Positive Data

Is it possible to build a propensity model (i.e., the likelihood that a user will buy an item) using only positive values. For example, I have a bunch of data about Customers (people that bought stuff)...
2
votes
1answer
31 views

What supervised machine learning model can be used to generate a scorecard-like result?

A scorecard is typically used in Credit Application. One very common model for developing a credit scorecard is logistic regression since it has well-defined probabilities. Apart from logistic ...
3
votes
4answers
162 views

ML: Classification Model Comparison

Given is a dataset that I need to use for a classification and I want to compare the performance of different classification models. Let's assume, I want to look at logistic regression (with ...