Questions tagged [predictive-modeling]
Statistical techniques used for predicting outcomes.
1,177
questions
0
votes
0
answers
7
views
find the parameter that minimize a multivariate distribution
i was trying to use scipy's minimize scalar to find the value of the parameter T that minimize the negative log-likelihood of a multivariate distribution with covariance matrix C. if i understood ...
- 101
0
votes
1
answer
15
views
How to cluster based on sensor data? - My first data science job
I'm on my first (real), data, programming job. As everyone can imagine, this can be quite hard and I learn a lot from it, given I am a data science student in university. However, I am completely ...
- 1
0
votes
1
answer
12
views
Do models of social systems suffer from prediction drift?
Background
I've created a binary classification model that predicts the probability of fraud for a given sample.
The choice of threshold allows me to set how many frauds are captured in the training ...
- 511
3
votes
1
answer
31
views
How can someone build a dataset for a "propensity to purchase" model?
Ok, this might seem a trivial question for some and it's not even a question, more like a discussion. I read the rules and I believe it's everything fine, so I'm gonna take my chances...
Here's the ...
- 31
0
votes
0
answers
24
views
Differential equations, real time measurements of variables, and ML
TL;DR: We measure variable $x$ every $10$ minutes, solve a differential equation $\frac{\mathrm{d}y}{\mathrm{d}t}$ where $y=f(x)$. We are interested in the time it takes for the cumulative value of $y$...
1
vote
2
answers
21
views
How can i implement an confucion matrix?
im trying to do a research but i need to make a confusion matrix how can i do that on this model?
https://www.kaggle.com/code/stpeteishii/race-classify-densenet201
Sorry im so so new to everything.
- 11
0
votes
0
answers
17
views
What are some approaches to improve the classification of ONE particular instance of interest?
I'd like to know if there are some methods to correct a specific misclassified instance of interest (e.g. in a Logistic Regression or Random Forest).
Like maybe increasing the error for that ...
0
votes
1
answer
54
views
How to implement a prediction model of machine learning when missing part of the data necessary for task required
I am relatively new to machine learning. I am trying to build a model that can predict the likelihood of an accident occurring on the road. I am currently using the FARS (Fatality Analysis Reporting ...
-3
votes
1
answer
33
views
What to do if your model's prediciton result wrong because of unlucky?
Have you ever had a situation where your model backtested with very good with historical data, and you also felt that your model was very logical? But when put it into practice case to predict the ...
0
votes
0
answers
38
views
Sklearn predicts different results depending on the input length
Here is the problem: I fitted a Random Forest Classifier and saved it to a pickle file. However, when I predict with the entire dataset I get one result, and when run predict line by line (loop) I get ...
- 1
0
votes
1
answer
13
views
How a Random forest "learns" or How loss (objective function value) is propagated back so that a random forest can "Improve"?
Every Blog and Youtube video talks about the same steps:
Choose that you have to build N number of tree and do the task 2-5 ...
- 323
0
votes
0
answers
9
views
Treat multiple periods of huge outliers in time series data with weekly seasonality data
How can I model a time series data (average sales is around 20K) with weekly seasonality that has multiple recurring outlier periods for example 4 days of huge volumes (around 150K) in March and 14 ...
- 153
0
votes
0
answers
3
views
how to define observation and event window
In a retail use case where I don't have a customer end date, so I tried to come up with a logic of taking the max difference within 1 year b/w consecutive purchases and based on a certain threshold ...
- 1
2
votes
1
answer
59
views
Is there a way to use debt details without aggregating them to predict the probability of payment of every debt per debtor?
I edited my post for clarity, for the second time. Thx lpounng for the feedback.
I am seeking advice on predicting debt payment within a year. Each debt has its own carachteristic wich are not easy to ...
0
votes
0
answers
14
views
What approach can be take to design a food menu intelligently?
I am given following problem, for which I am seeking for advice.
Problem Statement
Suppose you are given a catalogue of dishes with their cost and calories. Can you design an algorithm/model which can ...
- 197
0
votes
0
answers
12
views
Is it normal for a SARIMA model to produce no residuals?
I'm working on my first time series project where I am required to produce predictions for financial data.
The raw data is below:
Clearly, there is a seasonality and downward trend, I used the ...
- 1
2
votes
1
answer
37
views
Early anomaly detection / Failure prediction on time series
My problem here is that I want to predict failures in advance with respect to their occurrence. I have sensors mounted on my machine and with a certain frequency, they send data to my database. ...
- 45
0
votes
0
answers
30
views
Refining AI problem statement - suggestions
I am looking for some guidance. My company is a electronic goods manufacturing company. We work with multiple distributors (around 7 distributors) across specific regions to sell our products. But ...
- 2,449
0
votes
0
answers
17
views
Classification-Problem based on limited Dataset (Need keywords to search/ reading recommendations)
I have a dataset of about 200 test subjects, each with age, testscore and a one of two possible traits. I want to define some kind of function, where I input age and testscore, that predicts which of ...
- 1
0
votes
0
answers
13
views
How to make a predictive model using a timeseries data consisted of binary information?
I have a set of data that is showing the state of an object as a function of time. I would like to know what and how I should be utilizing machine learning to predict the state of the object at some ...
- 101
0
votes
1
answer
19
views
Which modeling technique is appropriate when I have nested/hierarchical data (individual and group) but user inputs will only be at the group level?
I am trying to create a predictive model that will be built on individual data, but user input will only exist at the group level. Reasoning is that I have 5 million rows of data at the individual ...
0
votes
0
answers
25
views
Can Hip-Hop/music trend be estimated?
How can I determine how an "evolutionary" music album affects the development of its genre? Only two perspectives I can come up with: 1. The effect on the number of songs before and after(...
0
votes
0
answers
15
views
How to handle dimensionality differences over time or between subjects
Note: This question has in mind tabular data, rather than imaging/NLP.
In the situation of collecting data over long periods of time, instruments may change and collect more precise data. This leads ...
-1
votes
1
answer
28
views
How to predict Total Hours needed with List as Input?
I am struggling with the problem I am facing:
I have a dataset of different products (Cars) that have certain Work Orders open at a given time. I know from historical data how much time this work in ...
- 19
0
votes
0
answers
25
views
Multi-label predictive modeling using Naive Bayes
I'm working on a predictive model that currently uses scikit-learn's Naive Bayes (testing with both Bernoulli and Gaussian) but I'm running into some trouble.
First I have a large dataset of records (...
1
vote
1
answer
40
views
Rule based vs predictive maintenance models
I have data for pumps which have one or more sensors to record the air pressure. Apart from the sensor_id and timestamp, with ...
- 185
0
votes
1
answer
49
views
Can a Simple Neural Network Predict a 0 or 1 Output by Looking Only at the Last Input?
I wrote a simple neural network that functions similarly to many of the C# examples I've seen online. It uses weights and biases and can be trained using backpropagation. It works well for ...
- 101
0
votes
2
answers
47
views
NLP vs Keyword-Search. which one is the best?
I have constructed a natural language processing (NLP) model with the aim of identifying technology keywords within text. The model is trained on a large dataset that contains over 400,000 phrases and ...
- 103
0
votes
2
answers
33
views
Predicting probability of reaching a milestone -- How much data should I use from production universe to train/test model?
If I am predicting probability of a business to reach (x) milestone (classification 1), but the only data I have is live production data, how much of the production data should I use to train the ...
1
vote
1
answer
59
views
revenue forecast using regression - what is the input for future?
I have a dataset with quarter wise revenue for past 3 years from Jan 2020 to Dec 2022. I have 4642 customers.
Each customer has 1 row of data which includes features based on his purchase frequency, ...
- 2,449
0
votes
0
answers
31
views
What is the formula to combine N correlated classifiers into single optimal one?
As we know if we train N probabilistic classifiers on same dataset, they will have some degree of correlation. As we also know, there is some method to assign optimal coefficients/factors/weights to ...
0
votes
0
answers
9
views
Neural Network for Food Prediction
I have been working on an app for a while now and now I’m up to the stage where I have to build a neural network for predicting user preferences based on some questions and personal data. Before I ...
- 1
0
votes
0
answers
15
views
Training Dataset preparation for customer Churn at a specific Month
I have dataset of customers from 2019-2022 . My goal is to predict customer Churn at a specific point in time , say exactly 3 months from the observation point ...
- 182
0
votes
0
answers
11
views
Find transistion matrix Markov chain
There is a two-state discrete-time Markov chain with a random variable:
$y_t = yx_t$ where $y = [1 \ 5]'$ (' is there because this matrix should be transposed).
It is known that:
$E(y_{t+1} | x_t) = [...
0
votes
0
answers
13
views
Which model is best to predict range from large vector of positive integers?
I'm creating a predictor, which takes a vector of table row counts (list of about 150 positive integers) and predict based on it the duration of the upgrade procedure (expressed as a set of ranges).
...
- 101
0
votes
0
answers
9
views
How to find confidence sets for a bunch of regression functions?
Unlike confidence intervals where we are interested in indicating a range of beta values in which the true parameter lies 95% of all times, I would like to understand how confidence sets are ...
- 23
0
votes
1
answer
24
views
Customizing Collaborative Filtering for Product Affinity
I'm trying to build a recommendation system and I am trying to use Collaborative Filtering (please let me know if other models fit better for my use case).
My Data:
My data is for an e-commerce site ...
2
votes
1
answer
45
views
How to implement linear regression
I am having difficulty achieving the same result as in sklearn while implementing linear regression model from scratch.
After adjusting the learning rate, I obtained an AUC of 0.694 for this binary ...
- 151
1
vote
0
answers
25
views
Are there any established approaches for dealing with a Degenerate Feedback Loop?
Scenario:
I develop a model which forecasts the likely sales success of a particular enquiry based on outcomes of past similar enquiries.
I then assign this likelihood score to new enquiries when they ...
- 111
1
vote
1
answer
20
views
Data Analytics Documantaions
I am working as a data analyst in a company. Me and my colleagues use different tools and software to analyze the data and make the reports (e.g., Excel, Python, R, Alteryx, SQL, Tableau).
Each one ...
- 1,905
0
votes
0
answers
28
views
What to use Multi-level Classification for Machine Learning predicting target Y1 and Y2
I have the data set where different signals are there. where it may be single or double or in groups so there are two outcomes where one outcome is dependent on the other.
...
0
votes
0
answers
24
views
High Recall and Low Precision for Binary CNN model
I was training a CNN model for binary classification. The training and validation accuracy seemed good. However, the precision is low and the recall is high (High false positive).
...
0
votes
1
answer
24
views
What kind of model should I use?
I am working with a dataset with about 10,000 customers. About 3,000 engaged with dozens of marketing campaigns over the years.
I am trying to create a model to find which marketing campaign to use on ...
- 3
0
votes
0
answers
14
views
Predicting multiple categorical targets: Single model vs multiple models
I have to predict multiple categorical columns. Each one is a multi-class classification.
Should I train separate models to predict each column individually or create a new categorical column which ...
0
votes
0
answers
5
views
How to predict multiple independent routes?
I have an idea in mind but, due to the lack of expertise in the ML domain, I just don't know where to start. I'd really appreciate any hints/advices on which methods to study or how to approach this ...
- 101
0
votes
1
answer
15
views
How to identify new potential customers from existing customers profile?
I have firmographic data of all the possible customers. Data includes sales, profitability, capital, organisation size, geographic location, industry, etc.
What is the best way of identifying new ...
- 233
0
votes
1
answer
25
views
how to proportionally round to integer values
I am trying to model student progress. Grades are integer values, but progress predictions are fractional.
For example, I have 10 students who are predicted a grade 6, but the model says they should ...
- 3
0
votes
0
answers
53
views
python Packages for correcting covariate shift
I am trying to build a regression model to predict customer revenue.
However, I see that my model almost always performs well on training and not on testing data.
While I am building a parsimonius ...
- 2,449
0
votes
0
answers
26
views
Clusters first then predict
Is it advisable for clustering techniques such as K-Means considered to be applied in generating target labels for prediction models?
- 161
0
votes
0
answers
16
views
a model to predict a day from other days?
I have the day of the month of 100 purchases made by a customer. Is it reasonable to use linear regression to predict the day of the month of purchase 101? Or what kind of algorithm should I use? How ...
