Data Science

Questions tagged [predictive-modeling]

Ask Question

Statistical techniques used for predicting outcomes.

821 questions
Newest
Active
Bountied
Unanswered
Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

Prediction for Current month based on last month's Labels

I have monthly data of loan installment repayment. The data contains basic features like salary,age,gender, credit score etc. Along with above features, i have data for last 6 installment failure/...
-1
votes
0answers
12 views

Tensorflow Model : Model.Predict Error “Matrix size-incompatible: In[0]: [1,20], In[1]: [21,4]” [closed]

Tensorflow Ver. 2.1 Model running with 21 columns/features. Learning and Test slices and normalized data. Model performs OK until try to use model.predict then ...
2
votes
2answers
55 views

Modelling data with Machine learning without a target variable

I am new to the data science community and wanted to understand the required steps that need to be taken into account while handling a dataset that does not have a target variable. I can do machine ...
4
votes
1answer
98 views

Calculate confidence score of a neural network prediction

I am using a deep neural network model to make predictions. My problem is a classification(binary) problem. I wish to calculate the confidence score of each prediction. As of now, I use ...
1
vote
0answers
13 views

Calculate marginal probability distributions of a dataset

I have a dataset with 'n' features and a label(binary) corresponding each entry. I am using a predicitive model to predict these labels using those 'n' features. Now, I wish to know the marginal ...
13
votes
4answers
1k views

In industry, what type of new data science algorithms does one develop?

I've seen several job descriptions for data science which include developing a novel algorithm to be a part of production environments. Can you give some input of what could be meant here exactly? ...
4
votes
1answer
22 views

Modifying a distribution by adding in samples incrementally

I would like to calculate the distribution (e.g., Gaussian) of a set of samples. However, I would also like to see how the distribution changes as I fit the samples into the distribution ...
2
votes
1answer
39 views

Machine learning solution approach to match loan repayments

I'm relatively new to the AI/ML space, but come from a programming background. The problem: I have a dataset of users transactions who have taken short-term loans from a single loan provider and I ...
3
votes
0answers
20 views

Improving a simple trig model

I have some data which I know is well approximated as a trig function, and I can fit it with scipy.optimize.curve_fit as follows: ...
1
vote
2answers
30 views

To find lookalike customers for Digital media targeting [closed]

I want to develop a model for digital marketing team. We have some set of ideal customers to whom we to send our promotions and have been converted. Now we want to pick past such cases and want to ...
2
votes
1answer
29 views

Find Net Reclassification Improvement/Index metric using Python

I am working on a binary classification problem with ~5k records and class proportion of 33:67. I have 60 features/variables in my dataset and finally I have come to about 10 variables based on ...
2
votes
1answer
31 views

How to evaluate performance of a new feature in a model?

I am working on a binary classification where I have 4712 records with Label 1 being 1554 records and Label 0 being 3558 records. When I tried multiple models based on 6,7 and 8 features, I see the ...
1
vote
1answer
28 views

Why results of statsmodel logreg is different from scikit-learn logreg?

I am trying to do a binary classification. I have only 6 input variables and one output variables. Label 1 is 1554 records and Label 0 is 3558 records. As you can see below, the metrics that I get ...
9
votes
4answers
335 views

How to impute Missing values not the usual way?

I have a dataset of 4712 records working on binary classification. Label 1 is 33% and Label 0 is 67%. I can't drop records because my sample is already small. Because there are few columns which has ...
1
vote
0answers
16 views

SVM C vs gamma hyper tuning

While running SVC(), how we can hyper tune C vs gamma combination? I could see changes in C and gamma are impacting the accuracy differently. Also what i understand about C and gamma are : 1) C is ...
2
votes
2answers
40 views

Kernel selections in SVM

I want to understand the kernel selection rationale in SVM. Some basic things that i understand is if data is linear then we must go for linear kernel and if it is non-linear then others. But ...
0
votes
1answer
22 views

Time Series Classification for loan data

I have multiple columns for loan installment repayment. As there is a field for month of repayment. I want to predict if the customer is going to pay next month installment or not. As I have ...
5
votes
1answer
54 views

Machine learning model bundled with a library vs. an API

I am thinking to "deploy" a machine learning model (in pickle it is sized 3 megabytes) and after discussing with my developer colleagues, they said it would be better if the model is packed as a ...
2
votes
0answers
24 views

Omnibus and R square improvements for OLS model

Checking on this community if any one can help on below problem posted by me on stats.stackexchange. https://stats.stackexchange.com/q/441653/266047 Detailed question is as below: ...
2
votes
1answer
26 views

Why and how to match variables in logistic regression?

I have a dataset of ~4.7K records focused on binary classification with 60 features. class 1 is of 1554 records and class 2 is of 3558 records. Now I would like to find the risk factors that ...
1
vote
1answer
33 views

Threshold to consider new feature as a new finding to a model?

I am working on binary classification problem with 5K records and 60 features. Through feature selection, I narrowed it down to 14 features. In existing literature, I see that there are well-known 5 ...
1
vote
1answer
36 views

ML Project - Achieve 2 Objectives

I have a dataset with 5K records focused on binary classification. I am posting it here to seek your suggestions on project methodology Currently what is my objective is 1) Run statsmodel logistic ...
2
votes
2answers
33 views

Which metric to choose for tracking model performance?

I am working on a binary classification problem with class proportion of 33:67. Currently what I am doing is running multiple models like LR,...
2
votes
1answer
43 views

How to select best feature set and not ranking for tree based models?

I am currently using feature selection approaches like filter, wrapper, embedded etc. All these methods give different set of ...
2
votes
2answers
311 views

Auto ML vs Manual ML for a project

I recently was introduced to a AUTO ML library based on genetic programming called tpot. Thanks to @Noah Weber. I have few ...
2
votes
1answer
263 views

feature selection using genetic algorithm in Python?

I have a dataset of 4712 records and 60+ features working on a binary classification problem. I already tried out all the feature selection approaches like ...
0
votes
1answer
41 views

1: 10 rule in logistic regression - EPV

I have a dataset with 4712 records. Label Yes - 1558 records and Label No - 3554 records. I read online that ...
2
votes
1answer
59 views

What is the difference between horizontal and vertical ensemble?

I am looking at different ways to do model ensembling and I came across the terms horizontal and vertical blending/ensembling but it is not well defined. My questions will be: What is the ...
2
votes
2answers
56 views

How to perform bootstrap validation?

I am working on a binary classification problem. I ran cross-validation and grid-search on train data. Later I validated the model on my test data as shown below ...
5
votes
3answers
52 views

Training on derived features that won't be present in a test set

So I have an odd real-world problem in which the data that's going to be fed to a categorical predictive model has (a few) certain features, and when building a model I can bolt on additional datasets ...
1
vote
1answer
20 views

How is DS used in the case of Payment Gateways?

I know it's a general question but what type of analytics can be done in this case? How can we apply machine learning models here?
3
votes
2answers
81 views

How to adjust cofounders in Logistic regression?

I have a binary classification problem where I apply logistic regression. I have a set of features that are found significant. But I understand that Logistic regression doesn't consider feature ...
2
votes
1answer
26 views

How to justify a predictor in influencing the outcome?

I am working on a prediction (binary classification) problem Currently I get an AUC score of 85-86 and F1-score of ...
1
vote
1answer
48 views

How to interpret shapley force plot for feature importance?

I am trying to practice and learn shapley value approach to explain my predictions on a binary classification problem. However am having difficulty in understanding the below plot. 1) Does it ...
1
vote
0answers
11 views

dense predictions time series

What do we refer to as "dense predictions" for time series ? I am reading through this Or to make dense predictions one step into the future, you might shift the features and labels by one step ...
6
votes
3answers
93 views

Regression: How to deal with positive skewness in continuous target variable

I'm working on a regression problem. My aim is to "learn" the distribution of a continuous target $y$ as good as possible to make predictions. My model looks like: $$y_i=\beta X_i + u_i.$$ $y$ is ...
0
votes
1answer
14 views

How to order the data with respect to data type

I am having large data set (82 variables) Is there any way to arrange data such a way that I have to get all numerical variables firstly then categorical variables so that I can run hypothesis ...
1
vote
1answer
39 views

How to choose input variables for ML

Let's say I have a huge database with 100K records and 60 columns. Let's say one of the column is "min_p". What I do is apply some logic/rule to determine the output label for this record. Basically I ...
2
votes
0answers
49 views

How to predict constant failing of equipment

I am trying to predict the failing of an equipment that heat up the liquid in a pipeline using a heat exchanger. The heat exchanger gets build up inside the pipe and thus needs to be flushed every ...
2
votes
2answers
49 views

Get the Polynomial Equation with Two Variables in Python

TL;DR predict "price", given "length" and "wandRate" I have some time-series data where the dependent variable is a polynomial result of 2 independent data points. Here is a snippet: This is ...
1
vote
1answer
33 views

Role of a known predictor in a disease prediction

I am working on finding out whether the patient will develop the disease or not in a hospital. Might be a basic info but I am just sharing it anyway. Usually through historic data, I was able to see ...
2
votes
1answer
317 views

How to interpret coefficients from logistic regression?

I ran a logistic regression (statsmodel) on my data with 60 features using the below code ...
0
votes
1answer
31 views

How to predict the dealer whether pick up the goods next month?

Here, I want to predict dealers(about 600) whether pick up the goods(about 30) next month. As you see, there are about 18,000 possibilities and it's difficult to predict. By the way, now I have ...
1
vote
1answer
32 views

How to forecast labels forward 3 years

I have many variables that I am using to predict median gross rent. I want to predict forward 3 years this is what my labels look like: How do I either preprocess the data in order to get a 3-year ...
1
vote
3answers
31 views

How to best use geographical information as a factor?

I am trying to predict crime rates and I have naively used lat and long as two separate factors (which seem to work well!). Are there any best practices for location as a factor?
1
vote
1answer
53 views

How to perform Permutation Feature importance?

I am trying to perform feature selection. Currently with Tree based classifiers, even randomly generated column is ranking above some of my real columns. So I was reading about ...
2
votes
2answers
69 views

Why SVM gridsearch takes longer time?

I have a dataset of 5K records and 60 features focussed on binary classification. Please find my code below for SVM paramter tuning. It's running for a longer time than ...
2
votes
2answers
77 views

How to yield better AUC score?

I have a dataset with 5K records and 60 features focused on binary classification. Class proportion is 33:67 Currently I am trying to increase the performance of my model which is stuck at F1-score ...

1 2 3 4 5 … 17