Questions tagged [model-selection]

Model selection is the process of comparing several models and their respective results to choose the model is best according to some evaluation metric.

Filter by
Sorted by
Tagged with
1
vote
2answers
39 views

How can I approach this problem?

Let's say I have a dataset with pricing information for the same flight during the past year. So, for a flight departing on day D, I have the available price from D-130 up to D (departure day). Then ...
2
votes
2answers
33 views

Is k-means with Mahalanobis a valid option for clustering?

I want more info into if k-means with Mahalanobis distance is a mathematically/methodologically correct option for datasets with different variance clusters. The steps are: Create aggregate datasets (...
2
votes
1answer
24 views

difference between empirical risk minimization and structural risk minimization?

I understand the meaning of empirical risk minimization as separate topic and was reading about structural risk minimization, it is hard for me to understand the difference between these two. I read ...
2
votes
1answer
39 views

Machine learning solution approach to match loan repayments

I'm relatively new to the AI/ML space, but come from a programming background. The problem: I have a dataset of users transactions who have taken short-term loans from a single loan provider and I ...
3
votes
0answers
20 views

Improving a simple trig model

I have some data which I know is well approximated as a trig function, and I can fit it with scipy.optimize.curve_fit as follows: ...
1
vote
1answer
20 views

Need of Weighted Mean Squared Error

We have MSE and RMSE as evaluation metrics for regression problems. I have for some problems people use Weighted Mean Squared Error (WMSE) as the evaluation metrix. Below is the WMSE formula: Can ...
1
vote
1answer
25 views

How can I compare my regressors?

I am trying to build a regressor for a dataset which gives info about students' school performance and the probability of getting admitted in the University of their choice. The first 5 observations ...
3
votes
2answers
55 views

How best to show the best model over multiple labels?

I have 4 models I trained and I want to display their prediction success over 45 different labels I tested them on. I get a very messy plot when I naively try to place them one on top of the other. ...
2
votes
2answers
36 views

What is my training score the mean_train_score or mean_test_score?

I am using sklearn to train some models (random forest, decision tree). For the training I am using RandomsearchCV with Stratified k-fold as cross-validation. Then I make a predictions on the test set ...
2
votes
3answers
34 views

Can someone explain what batch size is doing in convolutional NNs?

I've noticed that the performance of my models vary quite a bit as a function of the batch size, both in terms of the time to converge and (possibly) the amount of overfitting.I thought batch size was ...
1
vote
2answers
100 views

model with features of different sizes

I want to train a model (either classification or regression, doesn't matter) with features/inputs of different sizes, but I am not sure how to do it. For example, for each data-point, feature 1 and ...
0
votes
1answer
16 views

What are the steps and correct order of the operations in Machine Learning? [from Getting data to optimising models]

I've followed lots of tutorials on Machine Learning but in each of these, they go for a different strategy so it's quite confusing for me. I want to Know that what are the operations involved and what ...
1
vote
1answer
35 views

Association between features

Given the anonymized dataset of features below, where: "code" is a categorical variable. "x1" and "x2" are continuous variables. "x3" and "x4" are extracted features. They are the mean values of ...
1
vote
1answer
23 views

How to choose a model for this cross-validation curve?

I'm using GridSearchCV to tune hyperparameters for a Logistic Regression multiclass model. I read on Kaggle that you should choose the hyperparameter that results in the lowest discrepancy between ...
1
vote
1answer
29 views

Language modelling for Spell Checker

I am working on spell checkers, I want to create a spell checker, I am confused about which model to use Word-Level modelling Character-Level modelling plus I am preferring neural networks over ...
1
vote
1answer
71 views

Should scaling be done for mixed data (categorical and numerical)?

My dataset contains 13 attributes consisting of 10 Numerical and 3 Categorical attributes and Target. It has 180 observations ...
0
votes
1answer
20 views

Simplest way to build a semantic analyzer

I want to build a semantic analyzer i.e., to find how similar the meaning of two sentences are. For example- English: Birdie is washing itself in the water basin. English Paraphrase: The bird is ...
0
votes
0answers
6 views

Is there an ISO or other standard or best practice for structuring records related to organisations?

I'm working on a project which includes redesigning the way an application represents data about organisations. The current design is quite "flat" - meaning for example that if I want to add a new ...
1
vote
1answer
331 views

svm.LinearSVC: larger max_iter number doesn't always increase the accuracy/precision/recall

Background: Supervised machine learning Data shape 10+ features, target = 1 or 0 only, 100,000+ samples (so should be no issue of over-sampling) 80% training, 20% testing train_test_split(X_train, ...
0
votes
0answers
16 views

Model or algorithm for iterative optimization

Here is my problem : At every loop, I have new data that depends on the previous outputs. I need to approximate the function that optimizes (minimizes / maximizes) this new data on every iteration. ...
2
votes
2answers
72 views

What regression model can handle tiny amounts of data?

I'm trying to use machine learning to predict properties of a material during a crash test, but each data point requires physically crashing an expensive toy car, so I can only gather a few hundred ...
1
vote
1answer
45 views

Modeling strategy for predicting a day/hour based on my dataset

This is my first time posting here. I'm usually on SO. So I'm not sure if these kind of questions fit into DS stackexchange. I genuinely need opinions on this. What data do I have - ...
4
votes
1answer
50 views

help understanding nested cross validation

From what I read online, nested CV works as follows: I divide my whole data in k folds, where k-1 folds are the train set and one fold is the test set. There is the inner CV loop, where we may ...
0
votes
0answers
14 views

Do pseudo r squared metrics make sense for classifiers that aren't logistic regression?

I'm working with some domain scientists that are used to using logistic regression to predict a binary value. One of the ways they evaluate their logistic regression model is through the Nagelkerke $...
-1
votes
2answers
25 views

How to approach a problem when the information is in the relationship between the points, and not the points itself?

I am trying to analyze vehicular mobility models, where I am trying to learn how a particular vehicle moves and then detect similar patterns from the testing data. Here's what I have done for now: I ...
0
votes
0answers
24 views

Predicting house pricing given a dataset consisting of [ location: date of transaction: price ]

What would be the right way to tackle the problem of predicting median house pricing, given that the data I have for training consists in a big list of entries that have the following values: ...
2
votes
0answers
61 views

How to measure the stability of hyperparameter selection in a model-building procedure?

For my project I run several model-building-procedures. I use the mean and standard deviation of the test scores in the outer folds as an estimator for the generalizability of the model-building ...
3
votes
1answer
65 views

How to handle associated features in machine learning

I am working on a classification project in which some features are linked and I'm not sure how to handle them. I will simplify my project like that : There are different jobs, and multiple ...
0
votes
1answer
24 views

What model is recommended: I am using text features in a regression and want to interpret coefficients

I am using the text of comments on a forum to predict how many upvotes it will get. I want to be able to say, "Reviews with X, Y, Z words are more upvoted". So to do this, I want to use text features ...
0
votes
1answer
61 views

Building document classifier based on keywords, what would be the steps?

I have a requirement of classifying documents(.doc files) based on the profiles. I have a csv file with data: ...
0
votes
1answer
44 views

I have hourly data of a metric for 15 days, Can i predict the outcome values for same metric for the next 15 days?

I have tried a linear regression model for the same data, Since the regression line is continuous i'm not sure if it works to predict the outcome values for next 15 days, or for a given period of time!...
1
vote
2answers
266 views

What arguments should I pass to input_shape parameter of LSTM function in Keras?

My dataset has 2944424 rows and 6 columns. I am using an LSTM in Keras to forecast taxi demand. I am having problem with the input_shape parameter of the LSTM. It ...
1
vote
0answers
31 views

Using pipelines with a cross validation of several models in scikit-learn

Is there a simple way to cross-validate several models using sklearn pipelines?
5
votes
3answers
450 views

Why my network needs so many epochs to learn?

I'm working on a relation classification task for natural language processing and I have some questions about the learning process. I implemented a convolutional neural network using PyTorch, and I'm ...
0
votes
0answers
27 views

Model Selection

I have employed a variety of ML algorithms using 10-fold cross validation using the caret package in R on my data set. Can I employ ANOVA test on their f measures or Auc's to see if there is any ...
0
votes
0answers
16 views

Choosing predictive model for dataset

have a question about the type of model which I should use for a dataset I have. I have use 2 data-sets for my project. After hypothesis testing, I me Out of the 7 input variables, 6 of them are ...
0
votes
1answer
19 views

About 1M rows of data. Should I restrict myself to few columns as well?

I'm trying to build a predictive model from about 1 million rows of data. My goal is to predict a certain numerical value. I have the intuition that I should use very few numerical binary columns so ...
1
vote
2answers
45 views

What should I check if model accuracy is no better than baseline level(random guess)

I have a data with only 8 columns: id created_time employee_id rank position hourly price num_work_completed work_category hired Hired is the target variable with 1 representing hired and 0 ...
2
votes
2answers
107 views

Interpretation of ROC AUC score

i tried to evaluate 6 models and after plotting , this what i get : So i'm wondering , if those results are "Right" ? Thank's in advance.
0
votes
0answers
24 views

Cross validation for periodic signal in time series

I have a dataset of 90 periodic signals (current signals). These signals are divided into two big areas: Fault-free area and Fault area. I sliced the signals using a window of 80ms with an overlap of ...
0
votes
0answers
12 views

Many smaller city models or one country model?

I have a model selection question. I have models that predicts house prices. At Country level (with all the datapoints) the winner is a RandomForest with rsme 0.22 For a city level with many ...
0
votes
0answers
16 views

Predicting U.S. suicide count based on a set of inputs

I'm trying to design a model (or multiple) that can predict the number of U.S. suicides for a future year, based on a few inputs--"age", "sex", "population" (of the age/sex), and "gdp_per_year". I'm ...
0
votes
0answers
14 views

How do business policy rules/overrides work in large, production ML systems (esp. credit scoring)

so my startup has gotten to the stage where we are doing couple of 100k dollars per month. However our ML based credit scoring has become a jumble of a few hundred business policy rules with a ML ...
0
votes
0answers
136 views

Places365 for pytorch

I'm trying to use Places365 (the Vgg implementation) in PyTorch. I downloaded the model and the weights from the repo. The Vgg16 version of Places365 found in the official Github repo contains a ...
0
votes
2answers
51 views

How can I get an objective measure of song similarity?

I was browsing through ML project ideas and found an interesting one (just the problem statement ) which was: detecting if two songs are similar using lyrics. I found it to be an interesting idea but ...
0
votes
0answers
5 views

Will the combination of Chaid/CART tree modelling improve the accuracy of the Decision Tree Regression Model?

Will the performance of the decision tree regression model significanlty improve if we consider CHAID modelling first by identifying the key continous/categorical dependent variables and then builidng ...
0
votes
0answers
18 views

Is it possible to calculate precision/recall for a bag-of-words model?

Suppose I have a list of keywords given to a document: {keyword, extract, graph, represent, text, weight, number, document} and then I have the keywords ...
0
votes
0answers
18 views

Optimal architecture of neural networks for classification of samples with both text features and other features

Question: What is (from your experience) the most optimal architecture for a neural network for binary classification when the feature space is a mix of text and contextual features? Background: The ...
0
votes
1answer
32 views

Having averaged trials which are less than the number of features

Suppose I have an experiment where I have 70 features and 48 samples. The target variable is binary (0,1) and the 48 samples are divided such that 24 of them correspond to outcome 1 and the other 24 ...
2
votes
0answers
107 views

Test RMSE of polynomial regression drops when using more variables?

I am testing polynomial regression for a data set of 50 variables and a sample size of 5000. I ordered the coefficients of the linear model from high to low and then made different models using the p ...