Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [missing-data]

The tag has no usage guidance.

1
vote
3answers
39 views

What best/correct algorithm/procedure to cluster a dataset with a lot 0's?

I'm new to statistics so sorry any major lack of knowledge in the topic, just doing a project for graduation. I'm trying to cluster a Health dataset containing Diseases(3456) and Symptoms(25) ...
2
votes
2answers
19 views

Dealing with diverse groups in regression

What happens if a certain dataset contains different "groups" that follow different linear models? For example, let's imagine that examining the scatterplot of a certain feature $x_i$ against $y$ we ...
0
votes
2answers
43 views

what to do if the missing data in one column is based on some value/condition in another column in r?

I have a dataset with 20,000 observations and 19 variables. To start off with I have a gender column which has three levels namely 'M', 'F' and 'U' where U can be taken as not disclosed. Whenever ...
3
votes
1answer
24 views

How to deal with Missing Not at Random Data for k-means clustering?

I am running k-means clustering on a customer dataset. One of the available demographic fields is inferred homevalue, represented as an integer. This field has value 0 when it's inferred that the ...
2
votes
1answer
29 views

Cluster Analysis - Comparing Same Individuals Clustered Across Different Datasets with different features

I have an interesting problem, and I think my Google is failing me since I can't find the same problem anywhere. I have a set of individuals. I have 4 different datasets, with (some) to (all) of ...
1
vote
1answer
180 views

How to deal with missing data for Bernoulli Naive Bayes?

I am dealing with a dataset of categorical data that looks like this: ...
1
vote
1answer
115 views

Missing population values in census data

I have population data from Census.gov: Total US population by age by year from 1940 through 2010 Depending on the range of decades, the data is missing discrete population values for ages greater ...
0
votes
1answer
23 views

Handling NA Values in the Chicago Crime Rate data set

I am doing a little project on the Chicago Crime Rate data set and I noticed that there are over 600,000 NA values, primarily in the location fields. I feel that ...
0
votes
1answer
31 views

Missing Values In New Data

(Before someone marks this as duplicate - I'm not asking about training data, I'm asking about new data which has come in and needs to be classified) Suppose I've got a dataset which has 5 predictors ...
0
votes
1answer
1k views

To calculate unaffected part of the data set with missing values and positive skewness

A dataset has some missing values with positive skewness = 1. It is known that it is spread over 1.5 standard deviation from the median. How much % of data will remain unaffected?
0
votes
1answer
95 views

Orange imports numeric features as categorical (“file” widget)

Why are some of my numeric features not being recognized as 'numeric' types AND why can't I reclassify them? I can't share my CSV here but I can assure you those features are indeed numeric (I use ...
2
votes
0answers
25 views

Detect Missing Records in Dataset

I have a dataset that contains several measures from various widgets on a daily basis. While the widgets remain relatively stable over time, sometimes there are legitimate reasons for one to disappear ...
2
votes
0answers
117 views

Handling missing values to optimize polynomial features

I was playing around with some data to practice my Python and machine learning skills and wanted to create polynomial features from two features that I think are related and have a strong influence on ...
1
vote
0answers
17 views

How to incorporate an attribute that only exists in some observations?

In a binary classification problem, some of my observations have an event that occurs. I can, obviously, add a 1/0 flag if the event occurs ("event_occurred" in the data below). However, my intuition ...
1
vote
0answers
11 views

Finding features for missing classes

I have some data with labels from n-1 classes. There is no datapoint with the n-th class in the data but we know that data from the n-th exist. How can we generate data from the missing n+1-the class? ...
1
vote
0answers
18 views

Linear Model: How to deal with predictors with a lot of missing/small values?

I have a linear model used for prediction, with around 30 predictors, which are car usage rate as in percentage, across different zip codes. All these predictors have the same unit, as they are all ...
1
vote
0answers
39 views

Fix missing data by adding another feature instead of using the mean?

I am trying to build a model which predicts whether a user will unsubscribe from a service. There is a particular column which tells the number of hours until a report was written for the user. These ...
1
vote
0answers
508 views

predict() returns NA values

I have the problem. predict() method returns NA. My plan is: Read data from file and separate data to 2 sets: test and train Remove column with NA fraction over 95% Replace NA values with mean value ...
1
vote
0answers
18 views

dealing with dropped event data

I have a set of game play event data. One of our goals is to provide a picture of how much time the person has spent playing. This sounds simple but for the fact that events are often dropped and ...
1
vote
0answers
117 views

What are references explaining Hugo Steinhaus early “data science” work?

Historical background: Hugo Steinhaus can be considered as an early father of data science. He authored the paper Sur la division des corps matériels en parties, Bulletin de l’Académie Polonaise des ...
0
votes
0answers
11 views

Learning with missing features (MNAR)

I want to learn from features that may have some missing informations. The value of the variable that's missing is related to the reason it's missing (MNAR) To better understand my case, here is an ...
0
votes
0answers
5 views

Calculating target mean to validate if I should drop column with missing values is correct?

I am working on the KDD 2009 Cup Data Set (The Small one) and I have a question about preprocessing data. It has a lot of columns with null values, some of them have more than 90% of missing. Reading ...
0
votes
0answers
56 views

Finding unaffected data in a dataset

The data set has missing values. Further examination tells that they are spread along 1.5 standard deviation from the median with distribution mean = 0 & variance = 5. How much data (in percentage)...
0
votes
0answers
62 views

XGBOOST missing_value feature degrades my performance?

I'm training a XGBOOST model for gout disease on a training set I sampled 1-to-7 case-control ratio (enriched in cases). I have 220 features and I reach a cross-...