Questions tagged [data-science-model]
Questions about the organization of elements of data, and the standardization of their relations.
582
questions
-1
votes
0
answers
15
views
Which classification model to select
I have a classification problem where I need to identify whether a transaction will fail or succeed. The dataset is about 50,000 records. As per my understanding, using Naive Bayes classified, ...
- 3
1
vote
1
answer
18
views
How to Predict future temperatures based on past data with years
Hello everyone I am new to Machine learning and predicting. I just want to know if I can predict future temperatures based on past year's data and how I can do it Thanks. here is the pic of the ...
- 11
0
votes
0
answers
12
views
What is two-phase learning with random under sampling?
I came across this new technique of sampling which is using two-phase learning from amazon paper on Reducing Amazon’s packaging waste using multimodal deep learning
Can someone help in understanding ...
- 101
0
votes
0
answers
21
views
class label is less than 1 percent in classification problem
I am working on a ML problem where one class label is very less than even 1 percent. i.e 0.0002%
I have tried undersampling, oversampling, SMOTE but the results are not satisfactory on the model.
I ...
- 101
0
votes
1
answer
22
views
same random_state or mean of the different random_state?
As a part of my master's thesis, I am using different ML models for prediction and classification. The problem is I am confused if I should use only the result for a fixed random_state(suppose 10) or ...
- 3
0
votes
0
answers
10
views
Is it possible to get by month simulations in orbit?
I am looking to use Uber's Orbit (https://uber.github.io/orbit/) to get time series predictions.
By default, orbit will output point estimates with prediction intervals defined by the user. What i ...
- 1
4
votes
1
answer
415
views
How to Combat Data Drift
I have customer demographic data that include columns like: age, the first half of the postcode, occupation (there is a defined list of possible occupations), and more. Each month I get a new batch of ...
- 43
1
vote
1
answer
24
views
Modelling Player Impact on English Premier League
Let's say you have a very wide feature vector, all 0s and 1s, all 500 players in Premier League as features, out of which only 22 participate in a match. Those that participate are marked with 1s, ...
1
vote
1
answer
25
views
Building a simulation model for IoT device
I have an IoT device that takes inputs in bytes and returns the output in bytes; My goal is to build a software simulator for the IoT device.
Some example inputs & outputs:
...
- 11
0
votes
1
answer
13
views
What actually is model size scaling and how do i globally apply to every model?
I read this article on the EfficientNet paper and have seen a lot of this kind of scaling. For example, there's Tiny-YOLO, YOLO (the base),.. Some model like SVTR, people did scale it to Tiny, Small, ...
- 1
3
votes
1
answer
91
views
Taxonomy of train-test split approaches
I am looking for as close as possible for a exhaustive taxonomy of each train-test split approach.
For example, the 3 main splits that come to mind are:
A non-time based problem - would lead you to a ...
- 413
0
votes
1
answer
27
views
How to train a regressor model on data that has duplicate subjects but different records for each?
I am working on a dataset that is as follows (just an example):
prop_subj
prop_comp
bed_subj
bath_comp
sqft_subj
sqft_comp
A
B
2
1
1002
1006
A
C
2
2
1002
1075
A
D
2
2
1002
1000
B
G
2
1
1002
978
...
0
votes
0
answers
8
views
When to use "contextual" embeddings from language models?
When we use Word2Vec, it's obviously a non-contextual embedding because every word has the same representation. When I pass it to my LSTM, we say the hidden states are the contextual embeddings of the ...
- 109
0
votes
0
answers
12
views
Binary classification of video segment
Are there models that take a small video stream of 5-10 seconds as input (not a single frame, specifically video), and predict is there was an some kind of anomaly in this video segment or not?
0
votes
1
answer
51
views
Found input variables with inconsistent numbers of samples: ValueError
Today I am trying build ensemble model. Where I am working with iris dataset. In my model I am using ...
1
vote
0
answers
23
views
Queries on architecture of production pipeline for a batch predictions
I have a Problem statement to predict Expected delivery time (in days) and below is the information we have from client.
Prediction to be batch inference.
Each batch inference to have millions of ...
- 11
0
votes
0
answers
26
views
Irreversible Hash Encoding vs Reversible One-Hot Encoding
I understand that hash encoding for categorical features allows us to restrict or limit the dimensions of our dataset, which one-hot encoding does not.
I was wondering how we could compare these two ...
0
votes
1
answer
20
views
How to aggregate qualitative results from a simulation
I am trying to look for methods to aggregate and find the average of qualitative data that are outputted from a simulation.
There are 20 qualitative measures, each divided unevenly into 4 cycles ...
- 111
1
vote
0
answers
17
views
Working with massive data what is the right approach
let's say I have database with massive data (millions of rows)
additionally Let's say 26 Million rows are entered every day
I want to build a fraud model to check these 26 Million rows every day..
as ...
- 35
0
votes
0
answers
65
views
train_test_split error: Found input variables with inconsistent numbers of samples
I am trying to split a dataframe into training and test set and I am getting this error: "train_test_split error: Found input variables with inconsistent numbers of samples: [8819, 2795]".
...
- 11
0
votes
0
answers
5
views
Can I know if there's anyway to download Tableau report from server along with Title and Description in excel form?
I would like to download a Tableau report with the title footer and description written in Crosstab as it gets in PDF?
- 1
0
votes
1
answer
38
views
Why does my model's output look correct pattern-wise but with a consistent offset?
I'm having an issue where my AutoML model's output is having a consistent offset from the test dataset (image below). I'm wondering if anybody has any input on what could be causing this? My initial ...
- 101
1
vote
1
answer
16
views
Which performance metric is used for sequential dataset?
I have a dataset which looks like this, I have built LSTM model to perform seq prediction
...
0
votes
0
answers
8
views
Recommend low cost channel based on customers demographics
I am new to data science.
I am trying to implement recommendation system which only uses customer demographics data and activities like gender, age, customer transaction frequency in last 1,3 and 6 ...
- 121
1
vote
0
answers
38
views
modelling- Regression tree with constraints on depth and monotonicity
My goal is to build a regression tree model that takes as input (the z-score: the odds of the logistic regression) and a binary variable (default/non default) as target but with constraint on the ...
- 11
1
vote
2
answers
40
views
If both Train and test datasets are given in separate datasets. How to do model evaluation with train, validation, test?
Generally if one dataset is given we use
...
- 11
0
votes
0
answers
3
views
Behavioral Segmentation Clustering Mixed Large Data - with or without categorical variables
This question may be closed due to being too broad but I feel as though this is the best place to ask my question.
At the moment, I am dealing with some customer data and am looking on segmenting them ...
0
votes
0
answers
28
views
XGBoostError: [13:31:07] ../src/objective/regression_obj.cu:138: label must be in [0,1] for logistic regression
i'm tring tune XGBoost model for my dataset.
...
0
votes
0
answers
8
views
How to analyze a text and add weighted score to it
I have a set of keywords that I'm looking for in a string array. Is there a way to add weights/score based on the keyword and the tone, and calculate a score that can be used to compare. I'm not ...
- 1
2
votes
1
answer
46
views
How to group products that are similar and bought together over a short time window?
I have a retail dataset. I am trying to identify groups of products which are generally bought within a span of few days or on the same days (consider multiple store visits). For example - If someone ...
- 121
0
votes
0
answers
10
views
how to analyze the effect of currency and exchange rate on profit based on local currency and date
Yeah, the question is a mouthful, but basically, I have a reservations table that for our purposes looks something like this:
Keep in mind that this is a large table spanning many years and countries....
0
votes
0
answers
20
views
How Can I find the medical image datasets for congenital heart defect
I am working on implementing the image segmentation model for diagnosing a patient with a congenital heart defect.
But I couldn't find the image datasets.
please, anyone, help me to Find the Image ...
- 1
0
votes
0
answers
20
views
Demand Forecasting/Regression task for new products
I'm currently at the end of my master's degree and have to solve a data science problem. I am currently kind of stuck and need some kind of advice to get better results. I want to share the task I ...
0
votes
0
answers
15
views
Detecting anomalies/web performance on Google Analytics
I would like to detect any anomalies on my web traffic based on the data being collected by Google Analytics (GA).
Simply, I have form events data on GA and they normally have the same trend as ...
0
votes
0
answers
12
views
How to debug a neural network
After training wide& deep model for recommendation, weights of final layer becomes 0 leading to score of 0 for all inputs. I would like to know why this is happening and try weight initialization, ...
- 385
0
votes
0
answers
12
views
Walk-forward validation nrounds LightGBM
Dear Data Science Gurus,
I am facing the following phenomenon:
1-Scenario:
Predict demand for n-month in the future for hundreds of products using lightgbm. I have 2 years of history
2- Approach:
Walk-...
0
votes
0
answers
19
views
Customer Propensity Model Data Preparation, How to handle new customers?
I'm preparing a customer propensity model. I have five years of historical data. I plan to use a window of three years, i.e. customer behaviour for the last three years is used for predicting whether ...
- 231
0
votes
0
answers
10
views
Predicting exponentially distributed output with multi inputs with the help of neural networks
I want to construct a Neural network regression model for data (8 inputs and 1 output). My problem is that my output has exponential distribution. When I normalized it in 'log scale' or 'min-max scale'...
0
votes
0
answers
13
views
How to Predict Binary Classification problem having less dominant Features
I have following dataset.
Total 31 columns including Target.
Target column has value of either 1 or 0.
This is balanced dataset.
All 30 Feature columns also have value of either 1 or 0.
All these 30 ...
- 101
0
votes
0
answers
22
views
GNN Model - Analyzing Training Curve
Introduction. Actually, I am working on a Graph Neural Network (GNN) model to predict some graph-level float values. So, input=graph, output=float predicted value. I trained and evaluated the proposed ...
- 1
0
votes
2
answers
47
views
How can a log transformation decrease performance?
I'm working on a Demand Forecasting project, I have a lot of 0 (75% of the database)
I got a highly right skewed target (5.5).
So I decided to log transform my target: target = log(target + 1)
When I ...
- 1
0
votes
2
answers
35
views
Method of calculating sample size for a machine learning model [duplicate]
Is there any method of calculating sample size for a machine learning model/problem?
- 101
0
votes
0
answers
10
views
Datasets with Shared Features, Prediction on Dataset B using Shared Features from Dataset A + Extra Features
The title is not quite enough to describe the question well. Here is the general idea. I am working with a dataset $A$ with features $\{f_1,f_2, f_3, f_4\}$ and output $y$. I also have dataset $B$ ...
0
votes
0
answers
13
views
How to remove cross-sectional dependence from the residuals obtained from a non-parametric model?
I use the following model in R to model my data:
Y_it = f_it + e_it,
where I estimate the non-parametric trend (f_it) with a local linear smoothing procedure. Unfortunately, my residuals experience ...
- 1
1
vote
1
answer
83
views
What is the best machine learning technique to fuse two spatial data sets?
I have two data sets, containing points geometry (X,Y) and a recorded car exhaust parameter (let's say, RP value), of an area of ...
0
votes
0
answers
11
views
Multi-Output or Mult-Task Learning - Regression
ALl!
Problem Definition:
I have a dataset consisting of N samples (consider them 2D images), and there are K continuous output values(Y1, Y2, Y3, ...) for each sample. Actually, it is a regression ...
- 1
0
votes
0
answers
7
views
Application of eigenvalues, eigenvector, transposed matrix
Can you give me please some application of eigenvectors, eigenvalues and matrix transposition in data science?
I guess for eigen-values/vectors it would be linear regression PCA and NLP, alongside ...
- 113
0
votes
0
answers
8
views
How to measure consistency of members within a group?
Context: We run a collection of book clubs where members meet each week. Members are assigned to a specific club, but are free to join any club on any week. E.g., I may be assigned to the Tuesday club,...
2
votes
0
answers
63
views
Dealing with class imbalance in test set
I am building a machine that tries to predict which ISP customers will complain due to issues with the network. I am having some difficulties.
The idea is to use network metrics of ~300K customers as ...
- 133
0
votes
0
answers
16
views
How to validate Probabilisitc models when you only need the rankings of the output (Agent - Customer Pairing)?
A given dataset has call details, Agent specific and Customer specific features. The target output is Sale (1) or not (0). The data is highly imbalanced. Our goal is not to maximize number of sales ...
