Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [model-selection]

Model selection is the process of comparing several models and their respective results to choose the model is best according to some evaluation metric.

0
votes
1answer
11 views

About 1M rows of data. Should I restrict myself to few columns as well?

I'm trying to build a predictive model from about 1 million rows of data. My goal is to predict a certain numerical value. I have the intuition that I should use very few numerical binary columns so ...
1
vote
2answers
15 views

What should I check if model accuracy is no better than baseline level(random guess)

I have a data with only 8 columns: id created_time employee_id rank position hourly price num_work_completed work_category hired Hired is the target variable with 1 representing hired and 0 ...
2
votes
2answers
78 views

Interpretation of ROC AUC score

i tried to evaluate 6 models and after plotting , this what i get : So i'm wondering , if those results are "Right" ? Thank's in advance.
0
votes
0answers
16 views

Cross validation for periodic signal in time series

I have a dataset of 90 periodic signals (current signals). These signals are divided into two big areas: Fault-free area and Fault area. I sliced the signals using a window of 80ms with an overlap of ...
0
votes
0answers
12 views

Many smaller city models or one country model?

I have a model selection question. I have models that predicts house prices. At Country level (with all the datapoints) the winner is a RandomForest with rsme 0.22 For a city level with many ...
0
votes
0answers
16 views

Predicting U.S. suicide count based on a set of inputs

I'm trying to design a model (or multiple) that can predict the number of U.S. suicides for a future year, based on a few inputs--"age", "sex", "population" (of the age/sex), and "gdp_per_year". I'm ...
0
votes
0answers
13 views

How do business policy rules/overrides work in large, production ML systems (esp. credit scoring)

so my startup has gotten to the stage where we are doing couple of 100k dollars per month. However our ML based credit scoring has become a jumble of a few hundred business policy rules with a ML ...
0
votes
0answers
29 views

Places365 for pytorch

I'm trying to use Places365 (the Vgg implementation) in PyTorch. I downloaded the model and the weights from the repo. The Vgg16 version of Places365 found in the official Github repo contains a ...
0
votes
1answer
22 views

How can I get an objective measure of song similarity?

I was browsing through ML project ideas and found an interesting one (just the problem statement ) which was: detecting if two songs are similar using lyrics. I found it to be an interesting idea but ...
0
votes
0answers
5 views

Will the combination of Chaid/CART tree modelling improve the accuracy of the Decision Tree Regression Model?

Will the performance of the decision tree regression model significanlty improve if we consider CHAID modelling first by identifying the key continous/categorical dependent variables and then builidng ...
0
votes
0answers
9 views

Is it possible to calculate precision/recall for a bag-of-words model?

Suppose I have a list of keywords given to a document: {keyword, extract, graph, represent, text, weight, number, document} and then I have the keywords ...
0
votes
0answers
14 views

Optimal architecture of neural networks for classification of samples with both text features and other features

Question: What is (from your experience) the most optimal architecture for a neural network for binary classification when the feature space is a mix of text and contextual features? Background: The ...
0
votes
1answer
31 views

Having averaged trials which are less than the number of features

Suppose I have an experiment where I have 70 features and 48 samples. The target variable is binary (0,1) and the 48 samples are divided such that 24 of them correspond to outcome 1 and the other 24 ...
2
votes
0answers
30 views

Test RMSE of polynomial regression drops when using more variables?

I am testing polynomial regression for a data set of 50 variables and a sample size of 5000. I ordered the coefficients of the linear model from high to low and then made different models using the p ...
0
votes
0answers
14 views

Best model for Antimicrobial Resistance (AMR) prediction?

Some classes of problem are best solved by a specific class of machine learning model, due to the structure of the data (e.g. Deep Learning for computer vision). Prediction of bacterial resistance/...
1
vote
1answer
38 views

Probabilistic Machine Learning model to match spatial data

I have spatial data from multiple sources. This data consists of ID, lat, long, and time. My goal is that given a new lat-long, the model needs to return (preferably with a probability) the data ...
1
vote
0answers
24 views

I am getting p value 0 for my population I teven tried zscore for normalization

a1 is the columncolumn a1 g=scipy.stats.mstats.zscore(a1) g stat, p = normaltest(g) print "%.3f, %.9f" %(stat ,p) Result 8.570, 0.00000000 p1=scipy.stats.kstest(g,'norm') p1 KstestResult(...
2
votes
1answer
42 views

How to do vocabulary estimation based on observed writings?

Below is a scatter plot of the data set I am dealing with. The X axis is the total number of words per essay for a particular individual, and they Y axis is the number of unique words. In principle, ...
0
votes
0answers
53 views

Using logLoss as metric function for highly unbalanced dataset

ihave an highly unbalanced dataset and the caret pacjage only allows me to select accuracy or kappa as performance metric. Is it correct to use a mlogloss function to compute model performance? Do you ...
1
vote
1answer
38 views

Model selection: large mean and variance vs small mean and variance

This question was always in my mind. Imagine you are doing 5-10 fold cross validation and one model gives you mean accuracy of 0.8, but with 0.2 standard deviation and the other one gives 0.7 with 0....
2
votes
2answers
71 views

How to Work with Imbalanced Data

I am building a binary classifier from a set of feature vectors some of which are categorical like Yes or No (two options). I am replacing them with 1 and 0 and since there is strong imbalance between ...
2
votes
1answer
42 views

What metrics determine the quality of the model?

Working on this Kaggle competition, and have some questions. Using this code: ...
2
votes
2answers
83 views

Where does the “deep learning needs big data” rule come from

When reading about deep learning I often come across the rule that deep learning is only effective when you have large amounts of data at your disposal. These statements are generally accompanied by a ...
0
votes
0answers
38 views

Hyperparameter optimization when calculating learning curves

I'm selecting a model for a regression problem and want to calculate learning curves. My dataset consists of ~20,000 x-y pairs. I'm using kernel ridge regression with different kernels, different ...
2
votes
1answer
38 views

Ranking ATM based on Utilization and Economic Data (Scoring/Rank Model)

I have a sample data of around 10 ATM's Locations along with their Utilization Count (Deposits, Withdrawals and Others) for the past 3 months. I am planning to collect additional data such as nearby ...
0
votes
1answer
18 views

Tests to find which theory agrees better with observation

I have three curves ( 1.> observation: yobs , 2.> theory-1: yth1 , 3.> theory-2: yth2 ). All of these curves are functions of a single variable (say variable x.) From a computational perspective, all ...
1
vote
1answer
29 views

checking model stability - Performance for different class

I tried to do multi-class classification problem. The goal is to predict whether the match will be won by HomeTeam, AwayTeam or Draw. I did feature engineering from the attributes and finally came up ...
0
votes
1answer
40 views

Why some ML models can't take advantage of text ordering information?

On this google tutorial (https://developers.google.com/machine-learning/guides/text-classification/step-4) it is said: > Build n-gram model [Option A] We refer to models that process the tokens ...
3
votes
2answers
351 views

LSTM vs ARIMA for demand prediction

I'm new to the field of time series prediction. I'm looking for a demand prediction model to predict when the product will be sold out from the online supermarket (when the supply is known in advance)...
1
vote
2answers
108 views

How do you pronounce ROC?

In regards to ROC curves, how do you pronounce ROC? I have always spelled out the letters like R-O-C but I sat through a sales demo today where they guy pronounced it as a word like "rock" as in "the ...
0
votes
0answers
11 views

How to attribute variance to an input parameter?

Some data Maybe this is easiest to explain by going straight with the data. Here is how much money Bob has at the end of each day. ...
5
votes
4answers
254 views

Which is first ? Tuning the parameters or selecting the model

I've been reading about how we split our data into 3 parts; generally, we use the validation set to help us tune the parameters and the test set to have an unbiased estimate on how well does our model ...
0
votes
2answers
40 views

Does the choice of error function impact the model parametrs?

Suppose I have trained a multi variate linear regression model on a particular training set, and the model parameters $\theta=[\theta_1,\theta_2,\ldots, \theta_n]$ were determined by minimizing a cost ...
2
votes
1answer
17 views

P-value mining on large number of combinations of variables

I really don't know any machine learning, but have a problem that seems like one where I should use some ML algorithm. I am analyzing a medical study with one age-related condition, age, a treatment, ...
1
vote
1answer
34 views

Is the ultimate challenge in ML simply computational power?

I am stuck on a theoretical roadblock in learning about machine learning, because I have not seen this explicitly addressed anywhere. In my studies, it seems as if Cross-validation (or some variant ...
0
votes
1answer
155 views

How to Identify p (lag order) for ARIMA Model in Python

here is my auto correlation plot. Generated by the following python code. ...
1
vote
1answer
43 views

Nested cross-validation generalization error for multiple models

I am referring to this question: Nested cross-validation and selecting the best regression model - is this the right SKLearn process? In the answers it shows that nested cv can estimate the ...
3
votes
3answers
65 views

Folds in Cross validation

I am performing 10-folds cross-validation to evaluate the performances of a series of models (variable selection + regression) with R. I created manually the folds with this code. At the moment I'm ...
1
vote
0answers
30 views

Are there any Meta Knowledge bank available?

What resources do you use to learn meta knowledge ? By meta knowledge, I mean generalized information that will help us take more informed decisions when working on a problem later. Example of meta ...
0
votes
2answers
471 views

Why the VC dimension to this linear hypothesis equal to 3?

I am trying hard to understand this. Here is the scenario: X = R^2 H = { h(x) = x + 10 } I need to calculate the VC dimension for the above linear separator. ...
1
vote
0answers
37 views

Decent ROC, but horrible Precision-Recall curve

I was working on a model with following process: Split to training/validation/test sets Try a series of different models like GBM, RF, Logistic Regressions Optimize hyper-params on them using ...
1
vote
1answer
89 views

Optimizing decision threshold on model with oversampled/imbalanced data

I'm working on developing a model with a highly imbalanced dataset (0.7% Minority class). To remedy the imbalance, I was going to oversample using algorithms from imbalanced-learn library. I had a ...
0
votes
1answer
70 views

Which ML algorithm to use if we have categorical data, numeric data, derived data (derived from) other variable in our data set? [closed]

I am a beginner in Data Science. I have a data set which contains numerical data, categorical data and derived data (derived from other columns). The target column (dependent) is binary. Which Machine ...
1
vote
1answer
35 views

Which model may be best for outcome of a surgery?

New to data science and am trying to be a self-starter and implement advanced data analytics in my subspecialty of surgery. Below is a description of my data set. I know that I will have to explore ...
1
vote
1answer
44 views

Finding the equation for a multiple and nonlinear regression model?

Regarding nonlinear and multivariable regression, I use R or Matlab. In the case where a regression with just two variables, I simply draw the graph Y with respect to X, and look for the equation of ...
1
vote
1answer
94 views

can accuracy rise while precision and recall drop?

I am working on a model and running some experiments, I see that under some configurations, The accuracy rises while the recall and precision are much lower, what is the mathematical explanation? is ...
0
votes
1answer
85 views

Inference of root mean square value in terms of house prediction

The objective of the task is to predict the housing prices. A model is created based on California housing dataset to predict housing prices and is subjected to evaluation using the below code. ...
1
vote
1answer
63 views

Regression model for continuous dependent variable and count independent variables

I am currently learning R and I am relatively inexperienced in the field. Hope I can get some advice from you guys! I am working on a project where I have to estimate the average processing time of ...
2
votes
0answers
36 views

Please help to identify if there is meaning of linear variables separation by features in phase space?

Description: I have clustering problem with next input: I have truth dataset containing labels for each row of features. I've found that some features can be modified to some phase space, where they ...
2
votes
2answers
62 views

optimal combination of hyper parameters and model selection

This is a general question which often comes up when tuning deep learning and machine learning algorithms such as recurrent neural network, multilayer perceptron or SVM etc. When we tune the hyper ...