Questions tagged [data-science-model]

Questions about the organization of elements of data, and the standardization of their relations.

Filter by
Sorted by
Tagged with
-1 votes
0 answers
15 views

Which classification model to select

I have a classification problem where I need to identify whether a transaction will fail or succeed. The dataset is about 50,000 records. As per my understanding, using Naive Bayes classified, ...
  • 3
1 vote
1 answer
18 views

How to Predict future temperatures based on past data with years

Hello everyone I am new to Machine learning and predicting. I just want to know if I can predict future temperatures based on past year's data and how I can do it Thanks. here is the pic of the ...
0 votes
0 answers
12 views

What is two-phase learning with random under sampling?

I came across this new technique of sampling which is using two-phase learning from amazon paper on Reducing Amazon’s packaging waste using multimodal deep learning Can someone help in understanding ...
0 votes
0 answers
21 views

class label is less than 1 percent in classification problem

I am working on a ML problem where one class label is very less than even 1 percent. i.e 0.0002% I have tried undersampling, oversampling, SMOTE but the results are not satisfactory on the model. I ...
  • 101
0 votes
1 answer
22 views

same random_state or mean of the different random_state?

As a part of my master's thesis, I am using different ML models for prediction and classification. The problem is I am confused if I should use only the result for a fixed random_state(suppose 10) or ...
0 votes
0 answers
10 views

Is it possible to get by month simulations in orbit?

I am looking to use Uber's Orbit (https://uber.github.io/orbit/) to get time series predictions. By default, orbit will output point estimates with prediction intervals defined by the user. What i ...
  • 1
4 votes
1 answer
415 views

How to Combat Data Drift

I have customer demographic data that include columns like: age, the first half of the postcode, occupation (there is a defined list of possible occupations), and more. Each month I get a new batch of ...
1 vote
1 answer
24 views

Modelling Player Impact on English Premier League

Let's say you have a very wide feature vector, all 0s and 1s, all 500 players in Premier League as features, out of which only 22 participate in a match. Those that participate are marked with 1s, ...
1 vote
1 answer
25 views

Building a simulation model for IoT device

I have an IoT device that takes inputs in bytes and returns the output in bytes; My goal is to build a software simulator for the IoT device. Some example inputs & outputs: ...
0 votes
1 answer
13 views

What actually is model size scaling and how do i globally apply to every model?

I read this article on the EfficientNet paper and have seen a lot of this kind of scaling. For example, there's Tiny-YOLO, YOLO (the base),.. Some model like SVTR, people did scale it to Tiny, Small, ...
3 votes
1 answer
91 views

Taxonomy of train-test split approaches

I am looking for as close as possible for a exhaustive taxonomy of each train-test split approach. For example, the 3 main splits that come to mind are: A non-time based problem - would lead you to a ...
  • 413
0 votes
1 answer
27 views

How to train a regressor model on data that has duplicate subjects but different records for each?

I am working on a dataset that is as follows (just an example): prop_subj prop_comp bed_subj bath_comp sqft_subj sqft_comp A B 2 1 1002 1006 A C 2 2 1002 1075 A D 2 2 1002 1000 B G 2 1 1002 978 ...
0 votes
0 answers
8 views

When to use "contextual" embeddings from language models?

When we use Word2Vec, it's obviously a non-contextual embedding because every word has the same representation. When I pass it to my LSTM, we say the hidden states are the contextual embeddings of the ...
  • 109
0 votes
0 answers
12 views

Binary classification of video segment

Are there models that take a small video stream of 5-10 seconds as input (not a single frame, specifically video), and predict is there was an some kind of anomaly in this video segment or not?
0 votes
1 answer
51 views

Found input variables with inconsistent numbers of samples: ValueError

Today I am trying build ensemble model. Where I am working with iris dataset. In my model I am using ...
1 vote
0 answers
23 views

Queries on architecture of production pipeline for a batch predictions

I have a Problem statement to predict Expected delivery time (in days) and below is the information we have from client. Prediction to be batch inference. Each batch inference to have millions of ...
  • 11
0 votes
0 answers
26 views

Irreversible Hash Encoding vs Reversible One-Hot Encoding

I understand that hash encoding for categorical features allows us to restrict or limit the dimensions of our dataset, which one-hot encoding does not. I was wondering how we could compare these two ...
0 votes
1 answer
20 views

How to aggregate qualitative results from a simulation

I am trying to look for methods to aggregate and find the average of qualitative data that are outputted from a simulation. There are 20 qualitative measures, each divided unevenly into 4 cycles ...
1 vote
0 answers
17 views

Working with massive data what is the right approach

let's say I have database with massive data (millions of rows) additionally Let's say 26 Million rows are entered every day I want to build a fraud model to check these 26 Million rows every day.. as ...
  • 35
0 votes
0 answers
65 views

train_test_split error: Found input variables with inconsistent numbers of samples

I am trying to split a dataframe into training and test set and I am getting this error: "train_test_split error: Found input variables with inconsistent numbers of samples: [8819, 2795]". ...
0 votes
0 answers
5 views

Can I know if there's anyway to download Tableau report from server along with Title and Description in excel form?

I would like to download a Tableau report with the title footer and description written in Crosstab as it gets in PDF?
  • 1
0 votes
1 answer
38 views

Why does my model's output look correct pattern-wise but with a consistent offset?

I'm having an issue where my AutoML model's output is having a consistent offset from the test dataset (image below). I'm wondering if anybody has any input on what could be causing this? My initial ...
  • 101
1 vote
1 answer
16 views

Which performance metric is used for sequential dataset?

I have a dataset which looks like this, I have built LSTM model to perform seq prediction ...
0 votes
0 answers
8 views

Recommend low cost channel based on customers demographics

I am new to data science. I am trying to implement recommendation system which only uses customer demographics data and activities like gender, age, customer transaction frequency in last 1,3 and 6 ...
1 vote
0 answers
38 views

modelling- Regression tree with constraints on depth and monotonicity

My goal is to build a regression tree model that takes as input (the z-score: the odds of the logistic regression) and a binary variable (default/non default) as target but with constraint on the ...
1 vote
2 answers
40 views

If both Train and test datasets are given in separate datasets. How to do model evaluation with train, validation, test?

Generally if one dataset is given we use ...
  • 11
0 votes
0 answers
3 views

Behavioral Segmentation Clustering Mixed Large Data - with or without categorical variables

This question may be closed due to being too broad but I feel as though this is the best place to ask my question. At the moment, I am dealing with some customer data and am looking on segmenting them ...
0 votes
0 answers
28 views

XGBoostError: [13:31:07] ../src/objective/regression_obj.cu:138: label must be in [0,1] for logistic regression

i'm tring tune XGBoost model for my dataset. ...
0 votes
0 answers
8 views

How to analyze a text and add weighted score to it

I have a set of keywords that I'm looking for in a string array. Is there a way to add weights/score based on the keyword and the tone, and calculate a score that can be used to compare. I'm not ...
2 votes
1 answer
46 views

How to group products that are similar and bought together over a short time window?

I have a retail dataset. I am trying to identify groups of products which are generally bought within a span of few days or on the same days (consider multiple store visits). For example - If someone ...
0 votes
0 answers
10 views

how to analyze the effect of currency and exchange rate on profit based on local currency and date

Yeah, the question is a mouthful, but basically, I have a reservations table that for our purposes looks something like this: Keep in mind that this is a large table spanning many years and countries....
0 votes
0 answers
20 views

How Can I find the medical image datasets for congenital heart defect

I am working on implementing the image segmentation model for diagnosing a patient with a congenital heart defect. But I couldn't find the image datasets. please, anyone, help me to Find the Image ...
0 votes
0 answers
20 views

Demand Forecasting/Regression task for new products

I'm currently at the end of my master's degree and have to solve a data science problem. I am currently kind of stuck and need some kind of advice to get better results. I want to share the task I ...
0 votes
0 answers
15 views

Detecting anomalies/web performance on Google Analytics

I would like to detect any anomalies on my web traffic based on the data being collected by Google Analytics (GA). Simply, I have form events data on GA and they normally have the same trend as ...
0 votes
0 answers
12 views

How to debug a neural network

After training wide& deep model for recommendation, weights of final layer becomes 0 leading to score of 0 for all inputs. I would like to know why this is happening and try weight initialization, ...
  • 385
0 votes
0 answers
12 views

Walk-forward validation nrounds LightGBM

Dear Data Science Gurus, I am facing the following phenomenon: 1-Scenario: Predict demand for n-month in the future for hundreds of products using lightgbm. I have 2 years of history 2- Approach: Walk-...
0 votes
0 answers
19 views

Customer Propensity Model Data Preparation, How to handle new customers?

I'm preparing a customer propensity model. I have five years of historical data. I plan to use a window of three years, i.e. customer behaviour for the last three years is used for predicting whether ...
  • 231
0 votes
0 answers
10 views

Predicting exponentially distributed output with multi inputs with the help of neural networks

I want to construct a Neural network regression model for data (8 inputs and 1 output). My problem is that my output has exponential distribution. When I normalized it in 'log scale' or 'min-max scale'...
0 votes
0 answers
13 views

How to Predict Binary Classification problem having less dominant Features

I have following dataset. Total 31 columns including Target. Target column has value of either 1 or 0. This is balanced dataset. All 30 Feature columns also have value of either 1 or 0. All these 30 ...
  • 101
0 votes
0 answers
22 views

GNN Model - Analyzing Training Curve

Introduction. Actually, I am working on a Graph Neural Network (GNN) model to predict some graph-level float values. So, input=graph, output=float predicted value. I trained and evaluated the proposed ...
  • 1
0 votes
2 answers
47 views

How can a log transformation decrease performance?

I'm working on a Demand Forecasting project, I have a lot of 0 (75% of the database) I got a highly right skewed target (5.5). So I decided to log transform my target: target = log(target + 1) When I ...
0 votes
2 answers
35 views

Method of calculating sample size for a machine learning model [duplicate]

Is there any method of calculating sample size for a machine learning model/problem?
  • 101
0 votes
0 answers
10 views

Datasets with Shared Features, Prediction on Dataset B using Shared Features from Dataset A + Extra Features

The title is not quite enough to describe the question well. Here is the general idea. I am working with a dataset $A$ with features $\{f_1,f_2, f_3, f_4\}$ and output $y$. I also have dataset $B$ ...
0 votes
0 answers
13 views

How to remove cross-sectional dependence from the residuals obtained from a non-parametric model?

I use the following model in R to model my data: Y_it = f_it + e_it, where I estimate the non-parametric trend (f_it) with a local linear smoothing procedure. Unfortunately, my residuals experience ...
1 vote
1 answer
83 views

What is the best machine learning technique to fuse two spatial data sets?

I have two data sets, containing points geometry (X,Y) and a recorded car exhaust parameter (let's say, RP value), of an area of ...
0 votes
0 answers
11 views

Multi-Output or Mult-Task Learning - Regression

ALl! Problem Definition: I have a dataset consisting of N samples (consider them 2D images), and there are K continuous output values(Y1, Y2, Y3, ...) for each sample. Actually, it is a regression ...
  • 1
0 votes
0 answers
7 views

Application of eigenvalues, eigenvector, transposed matrix

Can you give me please some application of eigenvectors, eigenvalues and matrix transposition in data science? I guess for eigen-values/vectors it would be linear regression PCA and NLP, alongside ...
0 votes
0 answers
8 views

How to measure consistency of members within a group?

Context: We run a collection of book clubs where members meet each week. Members are assigned to a specific club, but are free to join any club on any week. E.g., I may be assigned to the Tuesday club,...
2 votes
0 answers
63 views

Dealing with class imbalance in test set

I am building a machine that tries to predict which ISP customers will complain due to issues with the network. I am having some difficulties. The idea is to use network metrics of ~300K customers as ...
  • 133
0 votes
0 answers
16 views

How to validate Probabilisitc models when you only need the rankings of the output (Agent - Customer Pairing)?

A given dataset has call details, Agent specific and Customer specific features. The target output is Sale (1) or not (0). The data is highly imbalanced. Our goal is not to maximize number of sales ...

1
2 3 4 5
12